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ABSTRACT 


U.S.  statutory  policy  requires  the  armed  services  to  continuously  balance 
manpower  inventory  with  congressionally  authorized  requirements.  Inaccurate 
forecasts  put  the  Navy’s  budget  at  risk  and  degrade  overall  mission  readiness. 
Navy  policymakers  must  be  able  to  rely  on  accurate  inventory  forecasts  to 
develop  necessary  manpower  plans  that  steer  inventory  to  match  planned 
authorizations.  Strength  planners,  in  turn,  rely  on  forecasting  models  like  the 
Officer  Strategic  Analysis  Model  (OSAM)  in  an  attempt  to  accurately  predict 
future  inventory  levels. 

This  study  utilizes  applications  of  data  farming  to  OSAM  to  simulate 
Unrestricted  Line  Officer  (URL)  inventory  over  a  seven-year  period.  Additionally, 
the  research  utilizes  applications  of  Design  of  Experiments  (DOE)  to  project 
Surface  Warfare  Officer  (SWO)  inventory  across  a  variety  of  assumptions, 
including  a  proposed  Enhanced  Probationary  Officer  Continuation  and  Re¬ 
designation  (EPOCR)  policy.  Analysis  finds  that  current  policy  will  reduce 
FY2016  URL  inventory  by  8%  over  a  seven-year  period,  and  over-execute  SWO 
inventory  authorizations  by  40%.  We  find  that  EPOCR  reduces  operating 
strength  deviation  (OSD)  in  total  SWO  inventory  strength  by  12%  by  FY2022. 
Additionally,  implementing  a  low  accession  plan  and  a  high  transfer  plan  is  the 
most  robust  in  correcting  OSD.  When  implemented  correctly,  EPOCR  has  the 
potential  to  decrease  OSD  to  modest  levels  with  minimal  risk  of  under-execution. 
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EXECUTIVE  SUMMARY 


U.S.  statute  requires  the  services  to  balance  their  manpower  inventory  to 
congressionally  authorized  requirements.  As  a  result,  Navy  strength  planners, 
community  managers,  and  policymakers  plan  and  manage  the  Navy’s  end- 
strength,  skill  inventories,  promotions,  and  accessions  to  meet  the 
congressionally  mandated  end  strength.  To  do  so,  they  must  continuously  and 
accurately  predict  the  Navy’s  inventory  strength  by  designator  and  paygrade 
throughout  the  fiscal  year.  Inaccurate  prediction  and  resulting  policies  put  the 
Navy’s  budget  at  risk  and  degrade  mission  readiness. 

Strength  planners  rely  on  the  accuracy  of  complex  simulation  models  to 
correctly  predict  the  Navy’s  future  inventory  strength.  Models  help  Navy 
leadership  understand  the  potential  impacts  and  risks  inherent  in  current  and 
planned  policies  to  the  force  inventory  structure. 

The  Officer  Strategic  Analysis  Model  (OSAM)  is  an  entity-based  stochastic 
simulation  model  that  allows  for  projection  of  annual  future  officer  inventory  and 
losses  across  communities  by  applying  various  historical  trends  and  force¬ 
shaping  policy  to  current  inventory.  The  current  version  of  the  model,  as  used  by 
strength  planners,  however,  is  stripped  of  its  stochastic  capability,  which  limits 
assessing  risk  in  the  model’s  results.  Additionally,  OSAM,  as  currently  used  by 
strength  planners,  simulates  only  one  scenario  at  a  time,  making  the  simulation 
model  ineffective  for  the  large-scale  implementation  necessary  for  investigating 
effects  of  manpower  policies. 

This  thesis  uses  concepts  of  design  of  experiments  (DOE),  data  farming, 
and  a  stochastic  enhancement  of  OSAM  to  forecast  Unrestricted  Line  (URL) 
officer  inventory.  Data  farming  is  used  to  simulate  100  replications  of  a  base- 
case  scenario  that  projects  Navy  Unrestricted  Line  (URL)  officer  inventory  over  a 
seven-year  period.  The  base-case  scenario  assumes  that  current  manpower 
policies  and  loss  rates  prevail  over  the  projected  period.  We  use  the  base  case  to 
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gain  insight  on  the  expected  trend  of  URL  inventory  strength  and  the  risks  in 
future  SWO  inventory  strength  assuming  the  Navy  continues  with  current 
manpower  policy.  We  find  that  total  operating  inventory  strength  falls  by  an 
average  of  8%  over  the  seven-year  period  from  24,815  officers  at  the  beginning 
of  2016,  to  22,890  ±  13  officers  at  the  end  of  2023.  The  mean  end  strength 
results  in  the  base  case  have  extremely  small  standard  deviation  values, 
suggesting  little  value  in  OSAM’s  stochastic  enhancement.  More  importantly, 
base-case  results  show  SWO  inventory  strength  will  over-execute  OPA  by  more 
than  40%  in  grade  03  over  the  entire  projected  period. 

Additionally,  the  study  uses  concepts  of  data  farming,  design  of 
experiments,  and  High-Performance  Computing  (HPC)  to  simulate  surface 
warfare  inventory  using  key  factors  and  assumptions,  including  a  proposed 
enhanced  Probationary  Officer  Continuation  and  Retention  (EPOCR)  policy.  We 
use  historical  loss  rate  categories,  accession  plans,  transfer  plans,  and  three 
user-added  force-shaping  plans  corresponding  to  EPOCR  losses.  The  current 
year  promotion  plan  is  used  and  an  auto-promotion  method  that  constrains 
inventory  to  the  Defense  Officer  Personnel  Management  Act  (DOPMA) 
guidelines  is  assumed.  Using  resources  available  at  the  Naval  Postgraduate’s 
Simulation  Experiment  and  Efficient  Design  (SEED)  Center,  a  Nearly  Orthogonal 
Latin  Hypercube  Design  for  three  discrete  factors  corresponding  to  EPOCR 
losses  is  crossed  with  a  Full-Factorial  Design  for  three  categorical  factors 
corresponding  to  the  historical  plans.  A  “landscape”  of  output  is  farmed, 
collected,  and  analyzed  for  insight. 

We  use  graphical  and  statistical  analysis,  meta-modeling,  and  robust 
analysis  to  gain  insight  from  the  results.  We  find  that  implementing  EPOCR 
significantly  reduces  grade  03  over-execution  by  FY2022,  and  levels  the 
average  operating  strength  in  all  grades  to  match  OPA.  Total  operating  inventory 
strength  is  reduced  by  12%  over  the  seven-year  period,  from  8,365  officers  at  the 
beginning  of  FY2016  to  an  average  of  7,375  ±  739  officers  at  the  end  of  FY2023. 
However,  we  also  find  EPOCR  greatly  elevates  the  risk  of  over-execution.  To 
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mitigate  this  risk,  we  use  robust  analysis  and  metamodeling  to  find  robust 
policies  that  control  OSD.  We  find  that  implementing  a  low  accession  plan  and  a 
high  transfer  plan  is  the  most  robust  in  correcting  OSD.  Finally,  we  find  when 
implemented  correctly,  EPOCR  has  the  potential  to  decrease  OSD  to  modest 
levels  with  minimal  risk  to  OSD. 
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I.  INTRODUCTION 


A.  PROBLEM  STATEMENT 

During  the  last  decade,  operating  inventory,  the  available  number  of 
officers  within  the  Navy’s  Unrestricted  Line  (URL)  Community,  has  continually 
exceeded  its  authorized  billets  (BA),  particularly  within  its  junior  pay  grades.  This 
phenomenon,  termed  by  Navy  inventory  strength  planners  as  Officer  Over- 
Execution  (OOE),  has  imposed  large  financial  costs  on  the  Navy’s  budget 
(Parcell,  2015).  Comparisons  of  operating  inventory  to  officer  programmed 
authorizations  (OPA),  the  authorized  end  strength,  for  four  URL  communities  are 
depicted  in  Figure  1.  Over-execution  is  particularly  profound  within  the  Navy’s 
Surface  Warfare  Community  (SWO).  The  SWO  community,  like  most  URL 
communities,  has  a  closed-loop  personnel  system.  This,  coupled  with  historically 
low  retention  at  the  mid-level  grades,  has  traditionally  led  to  more  SWO 
accessions  than  is  authorized.  However,  the  large  number  of  accessions, 
“apparently”  needed  to  sustain  the  low  retention  rate,  has  resulted  in  manpower 
operating  inventory  overages  in  the  junior  ranks  (Huff,  et  al.,  2015).  A  paygrade 
profile  for  the  SWO  community  comparing  operating  inventory  to  authorizations 
from  2012  to  2015  is  shown  in  Figure  2.  Grades  01  and  02  have  high  historical 
overages,  but  even  more  startling  is  the  over-execution  in  grade  03,  which 
exceed  40%  in  FY2014  and  FY2015.  If  left  uncorrected,  over-execution  is  likely 
to  persist.  However,  naively  fixing  over-execution  by  lowering  accessions 
introduces  risks  to  mission  readiness  as  required  billets  may  go  unfilled.  Any 
sound  solutions  require  accurately  projecting  and  predicting  future  operating 
inventory  based  on  the  inherently  unpredictable  future  behavior  of  personnel. 
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Figure  1 .  URL  Profile  Display  of  Operating  Strength  Inventory 

versus  Authorized  End  Strength 
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Figure  2.  SWO  Paygrade  Profile  of  Inventory  Strength  versus 
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As  part  of  their  long-range  planning  and  programming  tasks,  officer 
strength  planners  at  the  Navy  Strategic  Resourcing  Branch,  N100,  rely  on 
powerful  and  complex  manpower  inventory  models  like  the  Officer  Strategic 
Analysis  Model  (OSAM)  to  predict  long-term  trends  in  Officer  Inventory.  OSAM  is 
an  entity-based  deterministic  simulation  model  used  to  perform  annual  forecasts 
of  officer  inventory  over  the  Future  Years  Defense  Program  (FYDP).  Using 
OSAM  for  strength  projection,  however,  is  limited  to  single  small-scale  scenarios 
that  provide  limited  insight  to  strength  planners.  Additionally,  as  a  deterministic 
model  with  no  variation  in  its  results,  OSAM  does  not  provide  any  assessments 
on  the  risk  associated  with  its  results  (Saluke,  personal  communication,  April  26, 
2016).  This  thesis  enhances  OSAM’s  utility  by  introducing  randomness  to  the 
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model  and  by  using  proven  simulation  techniques  to  run  thousands  of  OSAM 
scenarios. 

B.  BACKGROUND 

The  Navy’s  manpower  costs  have  historically  constituted  a  large 
proportion  of  the  Department  of  the  Navy’s  budget  and  this  proportion  is 
expected  to  grow  over  the  next  fiscal  years.  The  military  personnel  budget  for 
active  naval  personnel  (MPN)  was  $29.35  billion  in  FY  2016.  This  constituted  the 
second-largest  appropriation  of  a  total  Navy  budget  of  $155.40  billion.  This 
amount  funded  manpower  requirements  of  54,333  officers  and  268,524  enlisted 
personnel  for  a  total  end  strength  of  322,857  (Department  of  the  Navy,  2016). 

Title  10  of  the  United  States  code  mandates  that  services  regulate  their 
inventory  to  authorized  end  strength,  the  number  of  uniformed  personnel  set  by 
congress  allowed  on  the  final  day  of  each  fiscal  year.  The  Defense  Officer 
Personnel  Management  Act  further  places  numeric  constraints  on  the  distribution 
of  active-duty  officers  in  grades  04  through  06  for  any  authorized  officer  end 
strength.  Additionally,  authorized  end  strength  must  be  balanced  to  budgetary 
fiscal  constraints.  This  responsibility  requires  a  concerted  effort  among  policy 
makers,  community  managers,  and  strength  planners  to  continuously  plan, 
manage  and  monitor  the  Navy’s  officer  accessions,  promotions,  transfers,  and 
skill  inventories. 

Steering  current  operating  inventory  levels  requires  accurately  projecting 
and  predicting  future  inventory  strength  using  current  and  future  policy  and  other 
assumptions.  The  Office  of  the  Chief  of  Naval  Operations  (OPNAV)  Manpower, 
Personnel  Training,  &  Education  (MPT&E)  Resource  Management  Division, 
Strategic  Resourcing  Branch,  N100,  is  responsible  for  projecting  current  Navy 
officer  inventory  to  end  strength  and  developing,  assessing,  and  recommending 
personnel  strength  management  policies  and  force-shaping  initiatives  that  will 
correct  officer  inventory  strength  to  authorized  end  strength.  Personnel  policy 
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derived  from  inaccurate  projections  may  lead  to  over-execution,  in  turn  affecting 
the  Navy  budget  and  mission  readiness. 

C.  NAVY  STRENGTH  MANAGEMENT 

Strength  management  concerns  aligning  operating  inventory  strength  to 
authorized  end-strength  through  manpower  management  policies  encompassing 
accessions,  promotions,  transfers,  and  force-shaping  initiatives.  This  section 
describes  terms  relating  to  Navy  strength  management  and  applicable  to  this 
study. 


1.  Manpower  versus  Manning 

Manpower,  also  referred  to  as  “spaces,”  drives  the  need  for  inventory. 
Manpower  encompasses  unconstrained  minimum  quantitative  and  qualitative 
positions  needed  to  fulfill  all  Navy  missions  and  functions.  Navy  resource 
managers  derive  manpower  requirements  by  translating  national  defense 
strategies  and  force  structure  policy  decisions  into  manpower  needs.  In  general, 
requirements  are  written  to  minimum  skill,  pay  grade,  and  quantity  for  each  unit 
or  Navy  platform.  However,  budget  constraints  further  obligate  policymakers  to 
decide  on  which  missions  and  associated  requirements  to  fund.  The  process 
ensures  that  sufficient  funds  have  been  set  aside  to  pay  for  “spaces”  when 
personnel  are  assigned  to  a  “space.”  The  number  of  authorized  requirements  is 
referred  to  as  end  strength.  Specifically,  end  strength  is  the  number  of  uniformed 
personnel  set  by  congress  allowed  on  the  last  day  of  each  fiscal  year,  and  is  the 
target  to  which  strength  planners  strive  to  match  inventory. 

Manning,  also  referred  to  as  “faces”  or  operating  inventory,  is  the  specific 
inventory  of  personnel  at  an  activity,  unit,  or  platform  in  terms  of  numbers, 
grades,  and  occupational  groups  or  skills.  Each  year,  the  Chief  of  Naval 
Operations,  through  the  Department  of  Defense  (DOD),  must  submit  proposed 
end-strength  to  congress  for  approval  and  funding.  Fiscal  constraints  on 
authorized  end  strength  and  how  well  the  Navy  manages  accessions, 
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promotions,  transfers,  and  to  some  extent,  losses,  directly  affects  the  inventory 
that  is  allocated  to  funded  requirements. 

2.  Operating  Strength  Deviation 

Fiscal  constraints,  manpower  management,  and  policy  create  differences 
between  authorized  end  strength  (spaces),  and  the  operating  inventory,  the 
number  of  personnel  available  to  fill  authorizations  (faces).  The  difference 
between  the  “spaces”  and  the  “faces”  is  what  this  study  refers  to  as  operating 
strength  deviation  (OSD).  Over-execution  is  deviation  when  operating  inventory 
strength  exceeds  authorized  end  strength,  the  converse  being  under-execution. 
Strength  planners  have  to  be  cognizant  about  operating  strength  deviation 
throughout  the  fiscal  year.  Deviation  in  end-strength  exists  in  both  enlisted  and 
officer  communities  and  is  tracked  by  skill  and  grade.  Once  deviation  has  been 
identified,  community  managers  implement  strength  management  policies  and 
force-shaping  plans  to  guide  future  inventory  levels,  or  strength,  to  authorized 
end  strength.  This  thesis  focuses  on  operating  strength  deviation  within  the 
officer  Unrestricted  Line  (URL)  community. 

3.  Navy  Strength  Planning 

Navy  strength  planners  are  tasked  with  determining  the  effects  that 
current  and  future  accessions  plans,  promotion  plans,  and  losses  will  have  on  the 
ability  to  balance  personnel  inventory  to  authorized  end  strength.  Officer  strength 
planners  predict,  plan  and  manage  total  gains  and  losses  by  skill  and  grade  for 
each  fiscal  year  to  meet  Officer  Programmed  Authorizations  (OPA),  the  number 
of  officers,  by  grade  and  designator,  programmed  to  meet  end  strength.  Strength 
planning  is  founded  on  predicting  losses  and  gains,  and  generally  follows  the 
equation 

Strength p  t  =  Strength p(M)  -  Losses p(l_u  +  Gains p(f_1} ,  (1.1) 
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where  p  and  fare  subscripts  representing  pay  grade  and  time,  respectively. 


Strength  planning  is  a  proactive  approach  that  can  identify  future  operating 
strength  deviations  and  provide  justification  for  policies  that  ensure  matching 
future  inventory  strength  to  authorized  end  strength.  Strength  planning  is, 
however,  a  delicate  and  complex  task  requiring  strength  planners  to  rely  on 
complex  personnel  inventory  strength  projection  models  such  as  OSAM. 

4.  Inventory  Strength  Modeling 

The  use  of  inventory  strength  projection  models  for  military  purposes 
dates  back  to  the  conception  of  military  operations  research  (Abrams,  1957). 
Navy  Strength  Planners  and  Manpower  analysts  use  these  models,  to  predict 
how  current  manpower  inventory  levels  evolve  over  time.  The  models  help  in 
assessing  the  risks  associated  with  current  and  planned  manpower  policies.  The 
Department  of  Defense  uses  various  inventory  strength  models  employing  varied 
inventory  projection  methods  and  techniques.  These  models  generally  follow  the 
projection  structure  of  Equation  1.1,  but  they  differ  in  purpose  and  design.  The 
following  sub-section  discusses  some  of  the  key  modeling  methods  with  relation 
to  OSAM. 


a.  Aggregate  versus  Disaggregate  Models 

DOD  aggregate  models  project  inventory  as  an  aggregate  of  inventory 
across  all  specialty  skills  (Schank  et  al.,  1997).  Disaggregate  models,  such  as 
OSAM,  further  partition  projections  by  individual  inventory  attributes  like 
occupation  or  skill,  years  of  commissioned  service,  and  pay  grade,  thus  providing 
more  detail  than  aggregate  models  (Schank  et  al.,  1997).  This  study  uses  OSAM 
to  project  URL  inventory  strength  by  designator,  pay  grade,  and  years  of 
commissioned  service. 
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b.  Short-Term  versus  Long-term  Models 

Short-term  inventory  strength  models  make  monthly  projections  within  a 
specified  period  in  a  given  fiscal  year.  Long-term  inventory  strength  models 
project  inventory  in  annualized  time  steps,  from  the  end  of  one  fiscal  year  to 
another  for  periods  longer  than  one  year.  OSAM  is  a  long-term  inventory  strength 
model  capable  of  producing  annual  inventory  forecasts  for  periods  beyond  seven 
years  (Schank  et  at.,  1997).  This  study  uses  OSAM  to  project  URL  officer 
inventory  from  FY  2016  to  FY  2023. 

c.  Deterministic  versus  Stochastic  Models 

A  model  is  deterministic  if  the  output  of  the  model  is  wholly  determined  by 
the  assumptions  and  input  settings  selected  by  the  user.  In  a  deterministic 
inventory  model,  each  unique  set  of  inputs  will  always  produce  the  same 
inventory  output,  which  makes  it  unnecessary  to  run  the  model  more  than  once 
for  a  particular  set  of  input  values  (Lucas,  2000).  Stochastic  models  have 
randomness  in  their  output  results.  OSAM  uses  pseudorandom  processes  in  its 
modeling  process  to  assign  losses,  promotions,  accessions,  and  transfers, 
inherently  making  a  stochastic  model.  Updates  to  the  latest  version  of  the  model, 
however,  removed  the  model's  random  characteristics,  effectively  making  it 
deterministic  (Mundy,  2014).  Using  OSAM  deterministically,  however,  subjects 
results  to  the  “flaw  of  averages,”  where  single  number  averages  are  fallaciously 
used  to  represent  uncertainty  (Savage,  2009).  This  study  utilizes  a  modified 
version  of  OSAM  that  reinstates  randomness.  Chapter  III  of  this  thesis  discusses 
this  modification  to  OSAM. 

d.  Static  versus  Dynamic  Models 

A  static  inventory  model  does  not  depend  on  the  time  factor  but  rather 

describes  the  inventory  at  a  specific  set  time  and  assumes  an  identical  flow  of 

inventory  over  time  (Law  &  Kelton,  2000).  Dynamic  models  such  as  OSAM 

project  inventory  from  one  period  to  another,  making  the  time  component  integral 

to  and  explicit  in  the  modeling  process  (Law  &  Kelton,  2000).  This  thesis  uses 
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OSAM  to  study  the  evoivement  of  Unrestricted  Line  Officer  inventory  strength, 
including  Surface  Warfare  Inventory,  over  a  seven-year  period. 

D.  PROPOSED  SOLUTIONS  TO  OVER-EXECUTION 

The  prevalent  over-execution  within  the  unrestricted  line  community,  and 
chiefly  within  the  SWO  community,  has  emerged  in  importance  and  focus  to 
Navy  leadership,  particularly  due  to  recent  budget  cuts.  As  such,  Navy  leadership 
tasked  N100  for  solutions.  One  such  solution  is  lowering  SWO  accessions  to  a 
level  that  is  just  enough  to  cover  authorizations  within  the  junior  ranks.  However, 
naively  fixing  over-execution  by  lowering  accessions  introduces  risks  to  mission 
readiness  as  future  mid-level  requirements  may  go  unfilled,  given  the  SWO 
community’s  low  retention  rates. 

Another  proposed  solution  is  the  Enhanced  Probationary  Officer 
Continuation  and  Re-designation  (EPOCR)  Board.  EPOCR  is  a  proposed  force¬ 
shaping  initiative  that  extends  retention  and  continuation  board  eligibility  to  SWO 
officers  with  four,  five,  and  six  years  of  commissioned  service  (P.  Saluke, 
personal  communication,  April  26,  2016).  The  Navy  uses  continuation  and 
retention  boards  to  separate  from  service  or  transfer  to  other  communities 
officers  who  no  longer  have  viable  progression  career  paths  within  their  parent 
communities.  The  SWO  community  typically  reserves  probationary  force-shaping 
policy  to  officers  with  less  than  four  years  of  commissioned  service.  As  proposed, 
the  policy  will  refer  to  the  EPOCR  board  up  to  120  officers  with  four,  five,  or  six 
years  of  commissioned  service.  This  study  investigates  the  impact  of  a  low 
accession  plan  and  the  EPOCR  policy  on  projected  SWO  inventory. 

E.  PURPOSE  AND  OBJECTIVE 

The  author’s  efforts  are  focused  on  utilizing  OSAM’s  inventory  projections 

capabilities  to  provide  a  quantitative  approach  to  understanding  the  risk 

associated  with  different  manpower  policies  to  the  future  composition  of  URL 

inventory,  including  the  Surface  Warfare  inventory.  Specifically,  the  study 

projects  FY  2016  URL  inventory  to  2023,  assuming  current  manpower  policy  and 
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loss  rates,  and  measures  the  deviation  of  operating  inventory  to  planned 
authorizations  by  FY  and  designator.  Additionally,  the  study  assesses  the 
implications  of  a  low  accession  plan  to  SWO  inventory.  The  EPOCR  policy  is 
also  investigated  to  determine  its  impact  on  SWO  future  operating  inventory. 
Stochastic  enhancement  to  OSAM  enables  us  to  produce  confidence  intervals  for 
projected  inventory  strength  results. 

F.  RESEARCH  QUESTIONS 

The  scope  of  this  thesis  is  guided  by  the  following  research  questions 
posed  by  N 100. 

1.  What  is  the  expected  short-  and  long-term  trend  in  URL  inventory 
strength  assuming  current  policy  and  prevailing  loss  assumptions? 

OSAM’s  dynamic  application  can  produce  inventory  evolvement  over  a 
specified  period.  Stochastic  enhancement  of  the  model  will  allow  construction  of 
confidence  intervals  on  inventory  strength  results. 

2.  What  are  the  long-term  risks  in  inventory  strength  for  the  SWO 
community  associated  with  current  policy? 

OSAM’s  disaggregate  methodology  allows  us  to  focus  analysis  on  an 
individual  designator.  Its  long-term  annualized  projection  allows  us  to  track 
inventory  strength  trends  for  periods  of  great  lengths. 

3.  What  is  the  impact  of  the  Enhanced  Probationary  Officer 
Continuation  and  Re-Designation  Board  on  SWO  inventory  strength? 

OSAM  is  flexible  to  the  extent  that  one  can  model  additional  force-shaping 
plans  using  its  force-shaping  option.  This  thesis  uses  OSAM’s  force-shaping 
option  to  model  losses  corresponding  to  officers  referred  to  EPOCR. 

The  insight  gained  from  answering  these  research  questions  will  identify 
risks  in  current  manpower  plans  and  provide  justification  for  adopting  sound 
policies  and  plans  that  will  guide  operating  strength  to  authorized  end  strength. 

Analyzing  the  effects  of  various  accession  plans  will  also  provide  an 
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understanding  of  the  effect  of  reducing  SWO  accessions  to  future  operating 
inventory  strength. 

G.  METHODOLOGY  AND  SCOPE 

This  thesis  combines  data  farming  and  the  efficiency  of  design  of 
experiments  (DOE)  with  a  stochastic  version  of  OSAM  to  obtain  analyzable 
data  from  the  model.  Applications  to  support  data  farming  and  the  stochastic 
version  of  the  model  are  developed  and  provided  by  the  Naval  Postgraduate 
School  (NPS)  Simulations,  Experiments,  and  Efficient  Designs  (SEED)  Center, 
a  research  center  within  the  school’s  Operations  Department  that  focuses 
on  enhancing  DOD  Research  via  simulation  design  and  analysis 
(https://harvest.nps.edu).  First,  the  OSAM  input  factors  that  contribute  the  largest 
variation  in  the  output  of  interest  are  identified  from  the  model,  and  with  guidance 
from  N100  manpower  analysts.  With  operating  strength  as  the  response  variable 
of  interest,  loss  rate  categories,  transfer  plans,  and  accession  plans  are  chosen 
as  the  explanatory  variables.  We  introduce  three  additional  force-shaping  factors 
for  SWO  inventory  with  four,  five,  and  six  years  of  commissioned  service  (YCS), 
respectively,  as  a  way  to  control  future  SWO  operating  inventory  strength.  The 
latter  represents  a  proposed  policy  that  increases  the  number  of  junior  SWO 
officers  referred  for  lateral  transfer  or  re-designation  via  the  Probationary  Officer 
Continuation  and  Re-designation  Board  (Borozny,  2015).  The  number  of  factors 
and  their  corresponding  levels  dictate  the  type  of  design  used  for  the  experiment 
since  complexity  is  limited  by  available  computing  power  (Kleijnen  et  al.,  2005).  A 
stochastic  modification  of  OSAM  is  adapted  to  utilize  multi-cluster  computing  and 
further  scripts  are  developed  to  consolidate  the  model’s  output  to  desired 
specification.  The  model’s  output  is  analyzed  using  advanced  statistical 
techniques  incorporating  regression  tree  analysis,  robust  analysis,  and 
metamodeling  to  explore  the  relationship  between  OSAM’s  input  factors  and 
projected  inventory  strength. 
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H.  BENEFITS  OF  RESEARCH 


OSAM’s  utility  as  an  inventory  strength  projection  tool  is  largely  untested, 
and  its  value  largely  underestimated.  This  research  extends  previous  studies  by 
Sibley  (2012),  Borozny  (2015),  and  DeHollan  (2015)  by  studying,  validating,  and 
improving  upon  OSAM’s  utility  as  an  inventory  projection  tool.  Insights  gained  will 
enable  manpower  analysts  to  effectively  use  the  model  to  accurately  project 
officer  inventory  strength.  The  methods  used  in  this  research  can  subsequently 
identify  the  true  effects  of  OSAM’s  inputs  on  end  strength  results,  and  measure 
risks  associated  with  current  and  planned  manpower  policies  to  future  inventory 
strength  profiles.  Additionally,  the  study  will  gain  insight  into  the  implications  of 
the  proposed  EPOCR  policy  to  future  SWO  inventory. 

I.  LITERATURE  REVIEW 

Human  resources  are  the  most  valuable,  yet  more  complex,  and  often 
most  expensive  asset  of  most  organizations.  Due  to  the  uncertainty  of  human 
behavior,  organizations  often  resort  to  simulation  models  to  emulate  their 
manpower  systems  and  predict  the  short-  and  long-term  composition  of  their 
future  personnel  inventory  strength.  A  large  amount  of  research  has  been 
conducted  to  support  the  development  or  improvement  of  manpower  models  for 
inventory  strength  projection  and  optimization  within  the  context  of  the  military. 
Sibley  (2012)  offers  an  extensive  summary  of  such  research.  Whereas  numerous 
manpower  models  exist  within  and  outside  of  military  application,  there  is  little 
research  that  applies  the  concepts  of  data  farming  and  design  of  experiments  to 
manpower  models.  In  fact,  Antony  (et  al.  2012)  finds  that  little  attention  has  been 
given  to  the  application  of  DOE  outside  the  manufacturing  context.  The  literature 
reviewed  in  this  section  focuses  on  some  of  the  few  studies  applying  data 
farming  and  DOE  to  military  manpower  models. 

Blosch  and  Antony  (1999)  apply  design  of  experiments  to  the  Rating 
Supply  Chain  Model  (RSCM)  to  study  risk  in  the  United  Kingdom’s  Navy 
manpower  planning.  Specifically,  their  study  uses  DOE  to  identify  key  variables 
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that  cause  gapping,  referring  to  a  job  that  is  not  filled  by  a  competent  and 
qualified  person,  while  on  sea  rotation.  RSCM  is  a  stochastic  queuing  inventory 
model  that  represents  the  flow  of  inventory  through  manpower  positions  and  their 
interrelations  with  events  such  as  recruitment,  promotion,  and  losses.  The  model 
takes  as  input  nine  covariates  and  outputs  distributions  on  queue  lengths  and 
waiting  times  within  the  queuing  process.  The  study  designed  an  experiment  to 
evaluate  the  effects  of  three  variables,  sea-shore  ratio,  length  of  sea-draft,  and 
billet  ratios,  and  their  interactions  using  one  of  Taguchi’s  orthogonal  array 
designs.  Results  from  the  experiment  allowed  for  analysis  using  analysis  of 
variance  (ANOVA)  and  other  statistical  techniques  to  determine  the  factors  that 
influence  gapping  at  sea.  Although  Blosch  and  Antony  were  conservative  in  the 
number  of  variables,  their  study  revealed  a  promising  application  of  DOE  to 
manpower  modeling  within  the  military  arena. 

Erdman  (2010)  applies  the  concepts  of  design  of  experiment  to  the  U.S. 
Army’s  Enlisted  Specialty  (ES)  model.  Erdman  uses  DOE  to  identify  the  variables 
with  the  greatest  effect  on  minimizing  the  deviation  between  projected  personnel 
inventory  and  authorized  end  strength.  ES,  like  OSAM,  is  a  dynamic  inventory 
model  that  projects  military  inventory  by  grade  and  skill  over  a  multi-year 
planning  horizon.  With  800,000  variables,  ES  adds  an  optimization  application 
with  224,473  constraints  to  the  inventory  modeling  process.  Erdman  evaluates 
52  objective  function  coefficients  of  the  model  that  place  the  greatest  weight  on 
decision  variables  and  he  uses  the  Plackett-Burman  (Plackett  &  Burman,  1946) 
approach  to  construct  his  experimental  design  (Erdman,  2010).  The  findings  from 
his  analysis  for  the  Army’s  G1  showed  a  possibility  of  a  14%  reduction  in 
deviation,  or  an  average  drop  of  8,355  misalignments  (Erdman  2010). 

Nelson  (2010)  uses  DOE  to  evaluate  the  Navy’s  Bureau  of  Personnel 
Metrics  and  Analytics  Branch  (BUPERS-34)  Reenlistment  model.  The 
reenlistment  model  uses  linear  regression  to  forecast  the  expected  reenlistment 
rate  for  the  Navy’s  prescribed  three  reenlistment  zones,  based  on  years  of 
completed  service,  assuming  current  conditions.  Nelson  uses  DOE  to  determine 
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the  most  influential  variables  in  the  model  that  predict  the  reenlistment  rate  for 
each  zone.  He  uses  a  3-factor,  2-level  full  factorial  design  for  each  zone  and 
analyzes  experimental  results  using  multivariable  regression  and  other  statistical 
techniques.  Nelson  identifies  a  number  of  insignificant  variables  in  the  model  and 
recommends  alternative  models  that  improve  robustness  and  fit  for  each  of  the 
three  enlistment  zones  (Nelson,  2010). 

The  thesis  research  of  Sibley  (2012)  provides  the  foundations  on  which 
this  thesis  is  based,  and  is  the  earliest  literature  known  to  this  author  utilizing  the 
concepts  of  data  farming  and  DOE  on  OSAM.  Sibley  seeks  to  evaluate  and 
improve  the  prediction  capability  of  an  earlier  version  of  OSAM  by  examining  the 
performance  across  the  simulation  model’s  loss  adjustment  factors  for  the  lateral 
transfer  variable.  With  90  factors  of  discrete  levels,  Sibley  uses  a  randomized 
Latin  Hypercube  (LH)  design  and  a  Nearly  Orthogonal  Latin  Hypercube  design  to 
vary  lateral  transfer  rates  and  loss  adjustment  rates  among  the  SWO  and  Human 
Resource  (HR)  officers  for  pay  grades  01-06.  Data  farming  and  DOE  allowed  for 
efficiently  exploring  a  factor  space  that  would  otherwise  have  been 
computationally  impossible  to  explore.  Sibley’s  work,  the  first  application  of  data 
farming  and  DOE  to  OSAM,  not  only  lays  the  groundwork  for  continuing  work,  but 
also  provides  proven  analytical  solutions  to  OSAM’s  limitations  adopted  in  the 
current  version  of  the  simulation  model. 

DeHollan  (2015)  advances  the  work  of  Sibley  (2010)  by  applying  concepts 
of  data  farming  and  design  of  experiments  to  an  updated  version  of  OSAM, 
enhanced  with  randomness.  DeHollan  measures  the  stochastic  variation  in  the 
enhanced  model  and  determines  the  effects  of  an  improving  economy  on  the  four 
largest  designators  of  the  URL  community,  specifically  focusing  on  pay  grades 
03  through  06.  His  work  differs  from  that  of  Sibley  (2010)  in  that  in  addition  to 
using  data  farming  and  DOE  to  validate  OSAM’s  behavior,  he  uses  the  model 
itself  to  analyze  behavior  and  future  composition  of  officer  inventory  based  on  a 
given  real-world  scenario.  Since  OSAM’s  existing  platform  does  not  have  the 
intrinsic  capability  to  model  economic  factors,  DeHollan  simulates  a  three-year 
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period  of  an  improving  economy  by  introducing  forced  losses  for  three 
consecutive  years.  He  uses  a  257-design  point  NOLH  design  to  vary  27  factors 
representing  forced  losses  per  pay  grade  per  designator  per  year.  He  analyses 
the  relationships  among  the  factors  from  the  results  of  his  experiment  using 
metamodeling  and  other  statistical  techniques. 

Borozny  (2015),  who  did  her  thesis  research  on  OSAM  concurrently  with 
that  of  DeHollan,  uses  data  farming  and  DOE  to  project  Navy  Surface  Warfare 
officer  inventory  over  a  six-year  period.  Using  the  same  modified  version  of 
OSAM  as  DeHollan,  Borozny  studies  the  interactions  among  OSAM’s  accession 
modeling  methods;  loss  rates;  and  an  aggregated  force-shaping  plan 
representing  an  EPOCR  policy.  Borozny  finds  that  OSAM’s  different  loss  rates 
cause  little  to  no  variation  in  the  model’s  end  strength  output,  holding  all  else 
constant.  Additionally,  the  loss  rates  resulted  in  counterintuitive  results  where 
higher  loss  rates  increase  end  strength  and  vice  versa.  Borozny  does  not  provide 
an  analytical  explanation  for  this  surprising  result,  but  this  author  suspects  that 
varying  accessions  by  accession  methods,  rather  than  by  accession  plans,  over¬ 
compensates  for  the  losses  in  end  strength  inventory  induced  by  the  loss  rates. 
The  current  thesis  research  differs  from  Borozny  (2015)  by  varying  the  accession 
plans  available  in  OSAM  rather  than  the  accession  methods.  Additionally,  this 
study  models  EPOCR  force-shaping  transfers  at  a  disaggregated  level, 
representing  each  of  the  three  affected  YCS  with  independent  factors. 

The  works  of  DeHollan  and  Borozny  provide  the  literature  that  serves  as 
proof  of  concept  for  augmenting  OSAM’s  inventory  strength  projection 
capabilities  with  data  farming  and  DOE  to  answer  questions  relating  to  officer 
manpower  policies,  force-shaping,  and  personnel  strength  planning. 

J.  NAVY  STRENGTH  PLANNING  MODELS 

Navy  strength  planners  and  policy  makers  use  a  variety  of  strength 
planning  models,  some  of  which  are  discussed  briefly  in  the  following  sections. 
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1.  Navy  Officer  Personnel  Planning  System  Model 

N100  currently  utilizes  the  Navy  Officer  Personnel  Planning  System 
(NOPPS)  model  for  its  officer  strength  planning  and  projection  purposes.  The 
model  is  used  in  conjunction  with  accession  and  promotion  models  to  develop 
officer  inventory  management  plans  that  will  guide  inventory  to  authorized  end 
strength.  NOPPS  uses  historical  loss  rates  and  average  inventory  to  predict 
monthly  inventory  across  the  Future  Years  Defense  Program  (FYDP).  The 
forecasting  utility  of  NOPPS  is,  however,  inadequate  and  limiting,  forcing  strength 
planners  to  calculate  forecasts  outside  of  the  model  (Frank,  1993). 

2.  Navy  Enlisted  Strength  Planning  Model 

The  Navy  Enlisted  Strength  Planning  (NESP)  model,  also  utilized  by 
N100,  is  an  event-based  deterministic  tool  used  to  forecast  the  Navy’s  enlisted 
end  strength  (DeSousa,  2015).  NESP  determines  enlisted  losses  for  a  given  year 
by  applying  historical  loss  rates  to  the  number  of  enlisted  sailors  eligible  to  leave 
the  Navy  within  that  year.  Much  like  NOPPS,  discussed  earlier,  NESP’s 
forecasting  ability  is  also  limited  in  that  strength  planners  need  to  calculate  loss 
rates  externally  (DeSousa,  2015). 

3.  Officer  Strategic  Analysis  Model 

The  Officer  Strategic  Analysis  Model  (OSAM),  utilized  for  this  study,  is  the 
primary  inventory  strength  projection  model  used  by  Navy  strength  planners  to 
project  long-term  annualized  projections  of  active  officer  inventory  strength  by 
pay  grade,  skill,  and  fiscal  year.  The  model  is  described  in  Chapter  III  of  this 
thesis. 

This  chapter  gives  an  overview  of  the  problem  that  is  the  focus  of  this 
research,  inventory  over-execution  within  the  URL  community,  and  describes  it 
within  the  context  of  inventory  strength  planning.  The  chapter  poses  the  specific 
research  questions  and  the  quantitative  methods  by  which  these  questions  are 
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explored  as  well  as  the  relevance  of  this  research.  The  next  chapter  discusses 
OSAM,  the  inventory  strength  simulation  model  employed  for  this  study. 
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II.  OFFICER  STRATEGIC  ANALYSIS  MODEL 


This  chapter  gives  a  basic  overview  of  OSAM,  including  a  general 
description,  the  model’s  design,  its  development,  and  a  general  description  of  the 
model’s  analytic  approach  and  processing  steps.  DeHollan  (2015)  gives  an  in- 
depth  description  of  the  model’s  development  and  architecture. 

A.  MODEL  OVERVIEW 

The  Officer  Strategic  Analysis  Model  (OSAM)  is  a  dynamic  entity-based 
long-term  simulation  model  used  to  project  annualized  personnel  inventory 
strength  by  skill  and  grade  over  a  user-specified  time  period.  The  model  is  one  of 
many  decision  tools  available  to  Navy  manpower  analysts  and  the  primary  model 
used  by  Navy  strength  planners  to  project  long-term  annualized  active  officer 
inventory  strength.  OSAM  simulates  inventory,  dynamically,  in  yearly  time-steps, 
by  applying  historical  loss  rates,  accession  plans,  transfer  plans,  promotion 
plans,  and  user-defined  forced  losses  to  the  beginning  of  year  (BOY)  inventory. 
The  model  applies  current  Defense  Officer  Management  Act  (DOPMA),  DOD, 
and  Secretary  of  the  Navy  (SECNAV)  policies  and  constraints  to  its  yearly 
inventory  strength  projections. 

B.  DEVELOPMENT 

OSAM  was  developed  in  2007  to  fill  a  Navy  void  of  decision  tools  capable 

of  projecting  active  officer  inventory.  The  model  was  developed  by  LMI,  a 

government-consulting  group,  specifically  for  the  Navy’s  N14  (Manpower, 

Personnel,  Training,  and  Education  Catalog,  2015).  As  of  the  date  of  this  writing, 

OSAM  has  not  completed  the  Verification,  Validation,  and  Accreditation  (VV&A) 

process  required  for  all  simulation  models  intended  for  DOD  use.  Nevertheless,  it 

has  undergone  extensive  upgrades  from  its  original  version  and  has  not  only 

contributed  to  effectively  supporting  planning  and  budgeting  of  long-term  officer 

inventory  strength  but  also  made  it  possible  to  convert  the  model  into  one  that  is 

data  farmable.  The  executable  application  of  OSAM  was  originally  built  in 
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Microsoft  Visual  Fox  Pro,  a  defunct  database  language  available  only  on 
government  computers,  limiting  OSAM’s  usefulness  (DeHoilan,  2015).  Another 
limitation  of  the  original  version  was  that  the  model’s  input  was  decentralized  to 
multiple  file  locations,  requiring  manual  adjustment  of  each  file  to  adjust  scenario 
settings,  a  tedious  process  that  was  prone  to  error  (Sibley,  2012). 

The  latest  software  version  is  built  in  Microsoft  Visual  Basic  (VBA)  and 
houses  its  data  files  in  Microsoft  Access.  The  transition  to  VBA  and  Access 
circumvents  government  restrictions,  expanding  OSAM’s  access  and  use  by 
Navy  strength  planners  and  manpower  analysts.  The  current  version  also 
consolidates  OSAM’s  input  data  to  a  single  database,  removing  the  need  to 
individually  edit  each  file  when  adjusting  scenarios  and  pushing  the  models 
toward  conforming  to  data  farming  standards. 

C.  SOFTWARE  ARCHITECTURE 

OSAM  follows  the  general  structure  of  a  typical  simulation  model.  The 
model’s  architecture  comprises  an  input  component,  a  simulation  module,  and 
output  component.  The  input  component  is  a  set  of  permanent  source  tables  built 
and  stored  in  MS  Access  files.  When  a  simulation  is  executed,  the  input  section 
links  to  the  module  comprising  a  user  interface  and  simulation  algorithms.  The 
interface  captures  and  transfers  user  input  settings  and  parameters  to  the 
simulation  module  that  drives  the  simulation  algorithms.  Each  scenario’s  inputs 
and  parameters  are  stored  in  text  files  outside  the  confines  of  the  model,  allowing 
for  easily  accessing  and  rerunning  or  modifying  scenarios.  After  each  simulation, 
the  model  saves  output  data  as  two  database  tables.  The  first  table  is  a  multi¬ 
year  summary  (MYS)  report  of  forecasted  inventory  data  pertaining  to  promotions 
selections,  loss  counts,  transfers,  and  accessions.  The  second  is  a  flow  point 
table  that  includes  detailed  promotion  data  including  the  number  of  vacancies, 
promotions,  and  flow  point  estimates  for  each  competitive  category  and  control 
grade  (04,  05,  and  06).  A  scenario  Analysis  Tool  built  in  Excel  is  included  as 
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part  of  OSAM’s  software.  A  representation  of  OSAM’s  interconnecting  systems  is 
shown  in  Figure  3. 


Scenario 

File 


Figure  3.  OSAM  Database  and  Model  Component  Structure 

D.  SIMULATION  MODELING  OVERVIEW 

OSAM’s  simulation  process  is  similar  in  design  and  structure  to  Equation 
1.1  displayed  in  Chapter  I  of  this  thesis,  expressing  the  relationship  between 
beginning  inventory,  losses,  accessions,  and  the  end  strength.  The  analytical 
process  begins  with  a  beginning  of  year  (BOY)  inventory  that  accounts  for  each 
active  duty  officer  brought  forward  from  the  previous  fiscal  year  (FY),  and 
represented  by  pay  grade,  designator,  year  of  commissioning,  and  date  of  rank. 
The  model  subtracts  inventory  from  BOY  based  on  forecasted  losses  and  user 
specified  losses,  and  applies  accessions  and  promotion  algorithms  to  determine 
the  end  of  year  (EOY)  inventory.  For  a  multi-year  forecast,  EOY  inventory  for  the 
previous  year  is  the  BOY  inventory  for  the  subsequent  year.  Since  OSAM  is  a 
dynamic  long-term  disaggregate  model,  this  process  is  repeated  for  each  year  for 
each  designator  and  pay  grade,  for  all  years  defined  by  the  user.  Figure  4  shows 
an  overview  of  OSAM’s  analytical  process. 
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E.  OSAM  MODELING  PROCESSES 

This  section  discusses  how  OSAM  models  losses,  promotions,  accessions 
transfers,  and  accessions  to  forecast  officer  inventory 

1.  Losses 

Forecasting  losses  represents  the  most  complex  aspect  of  personnel 
inventory  strength  projection  and  is  the  first  analytical  task  of  OSAM’s  modeling 
process.  Navy  officer  strength  planners  categorize  a  loss  as  either  natural  or 
forced,  depending  on  whether  the  loss  was  deliberately  contrived  through  a 
force-shaping  policy.  OSAM  categorizes  losses  in  a  similar  way  as  Natural  losses 
and  Force-out  losses  and  adds  an  option  to  define  user-added  losses. 

a.  Natural  Losses 

Officer  natural  losses,  strictly  speaking,  comprise  losses  other  than  those 
due  to  force-shaping  policies.  These  include  officers  who  discontinue  their 
service  through  voluntary  retirement,  resignation,  and  discharge.  The  natural 
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losses  in  OSAM  are  an  aggregate  of  natural  losses,  losses  due  to  other  than 
force-shaping  policy  and  behavioral  losses.  OSAM  derives  natural  losses  using 
historical  loss  rates  predetermined  by  the  Navy  Officer  Personnel  Planning 
System  (NOPPS)  Forecasting  Model  (NFM).  NFM  estimates  loss  rates  by  FY, 
loss  types,  designator,  paygrade,  and  years  of  commissioned  service  (YCS) 
group.  OSAM  models  natural  losses  by  multiplying  the  historical  loss  rates 
derived  from  NFM  by  the  beginning  of  year  (BOY)  inventory  as  denoted  as 

Losses Proj  ( g,d)  =  LossRate (g,d)x  InvBOY  ( g,d )  ^  ^ 

where  LossesPr0j  (g,d)  represents  the  projected  number  of  losses  in  pay  grade  g 
for  a  specified  designator  d.  LossRate  represents  the  applied  loss  rate,  and 
InvBov  is  the  inventory  at  the  beginning  of  the  projection  year. 

b.  Force-out  Losses 

Forced  or  Force-out  losses  are  driven  by  force-shaping  plans  which  are 
policy  decisions  stipulated  by  law  and  Navy  leadership  to  compel  personnel  to 
leave  service.  The  primary  force-shaping  plan  for  Navy  officers  is  higher-year 
tenure,  which  limits  the  number  of  years  of  commissioned  service  at  any  specific 
pay  grade.  Navy  promotions  occur  within  specified  YCS  groups,  and  the 
percentage  of  officers  promoted  within  each  group  must  fall  within  minimum  and 
maximum  percentages  established  by  law  (SECNAVINST,  2006).  The  flexibility 
to  balance  promotions  with  higher  year  tenure  creates  an  effective  force 
management  tool  for  Navy  policy  makers  and  officer  strength  planners  that  can 
be  used  to  shape  the  inventory  composition  of  any  officer  community.  OSAM 
accounts  for  forced  losses  by  applying  maximum  YCS  conditional  constraints  to 
each  pay  grade  designator  combination: 

[g  >  4  AND  YCS  >  YCSretire  (g,d)]OR[g  <  3  AND  YCS  >  YCSMAX  (*,</)] 

where,  for  each  officer,  YCS  represents  years  of  commissioned  service  group, 
Y CSjyjax  is  the  maximum  allowable  YCS  group  for  grade  g  and  designator  d,  and 
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YCSRETiRE  represents  the  maximum  allowable  YCS  group  allowed  prior  to 
retirement  (Mundy,  2014). 

c.  User-Added  Losses 

In  addition  to  natural  and  forced-out  losses,  OSAM’s  flexibility  allows  a 
user  to  add  additional  loss  constraints  by  either  adding  or  reducing  losses  by  pay 
grade,  designator,  YCS,  or  FY,  using  the  force-shaping  utility.  The  model  adds 
user-added  loss  constraints  by  adding  or  decreasing  the  number  of  losses  after 
force-out  and  natural  losses  have  been  applied  to  BOY  (Mundy,  2014).  One  can 
specify  user-added  loss  constraints  by  FY,  pay  grade,  YCS  group,  designator, 
and  community.  For  instance,  selecting  to  increase  losses  by  20  applies  20 
additional  annual  losses  in  each  FY  of  a  specified  year  range,  and  distributes  the 
losses  within  the  specified  grades,  YCS,  and  designators  specified.  OSAM 
applies  the  losses  and  additions  stochastically  using  distributions  proportional  to 
inventory  at  the  BOY  of  the  specified  FY.  This  thesis  uses  OSAM’s  force-shaping 
utility  to  enforce  user-added  losses  representing  SWO  officers  referred  to  the 
EPOCR  board. 

2.  Promotions 

The  second  step  in  OSAM’s  modeling  process  is  determining  and  setting 
promotions.  OSAM  models  promotions  in  two  phases,  representing  control-grade 
and  fully-qualified  promotions. 

a.  Control  Grade  Promotions 

Control-grade  promotions  apply  to  grades  04  to  06.  Promotions  to  these 
grades  is  subject  to  stipulations  in  DOPMA,  which  provides  limits  on  the 
authorized  end  strength  and  the  minimum  time  in  grade  or  flow-points  in  each  of 
these  grades.  DOPMA  also  establishes  promotion  zones.  Zones  are  correlated 
with  an  officer’s  time  in  grade  and  officers  eligible  for  promotions  are  categorized 
as  either  “in  zone”  or  “above  zone,”  depending  on  whether  they  have  already 
been  considered  for  promotion.  DON  guidelines  further  regulate  promotion 
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percentage  rates  to  these  grades  (SECNAVINST,  2006).  All  naval  officers 
compete  for  promotions  with  other  officers  of  the  same  competitive  category, 
regardless  of  their  designator.  Surface  Warfare,  Submarine  Warfare,  Naval 
Aviator,  and  Naval  Flight  Officers,  the  designators  that  are  the  focus  of  this  study, 
belong  to  the  Unrestricted  Line  (URL)  competitive  category.  The  designators 
compete  against  one  another  for  available  promotions  by  grade  at  each  selection 
cycle. 

OSAM  provides  four  methods  for  modeling  promotions.  The  first  models 
promotions  based  on  a  predetermined  promotion  plan.  The  second  sets 
promotions  according  to  predicted  vacancies.  The  third  method  promotes  to  the 
number  of  vacancies  or  until  flow-points  reach  minimum  thresholds.  The  fourth, 
“Auto-Promote,”  promotes  to  the  number  of  vacancies  while  remaining  within 
DOPMA/DOD/SECNAV  guidelines.  This  thesis  uses  the  FY2016  promotion  plan 
to  project  FY2016  and  FY2017  BOY  inventory,  and  “Auto-Promote”  to  model 
subsequent  years. 

b.  Fully  Qualified  Promotions 

Fully  qualified  promotions  refer  to  officer  promotions  to  the  grade  of  02 
and  03.  Nearly  all  eligible  officers  will  promote  to  these  grades.  OSAM 
automatically  promotes  officers  to  these  grades  without  the  need  for  user  input. 

3.  Transfers 

The  third  modeling  process  applies  transfer  rates  to  inventory.  A  transfer 
refers  to  the  reassignment  of  an  officer  from  one  designator  to  another.  Naval 
officers  may  transfer  by  way  of  upward  mobility  from  a  training  designator  to  a 
primary  designator,  or  laterally  through  a  lateral  transfer  or  re-designation  board. 
OSAM  applies  upward  mobility  transfers  from  a  training  to  parent  designator 
automatically  without  user  input.  The  model  applies  lateral  transfers  based  on  a 
user-specified  transfer  plan.  Additionally,  the  model  constrains  transfers  to  only 
grades  01-06  and  mimics  historical  transfer  patterns  by  YCS  and  pay  grade. 


25 


4. 


Accessions 


The  last  step  of  OSAM’s  modeling  process  is  applying  accessions  or  gains 
to  obtain  the  end  of  year  (EOY)  inventory.  An  accession  refers  to  personnel  entry 
into  the  Navy  manpower  system  from  the  civilian  sector.  OSAM  uses  accession 
plans  that  are  based  on  historical  data  of  prior  accession  plans  executed  by  the 
Navy.  The  plans  specify  the  number  of  officer  accessions  by  community  and 
paygrade.  Alternatively,  users  can  choose  either  unconstrained  or  constrained 
accession  methods  to  model  accessions  for  each  projected  FY.  The 
unconstrained  method  sets  the  number  of  accessions  so  that  approved  end 
strength  is  met  or  exceeded.  The  constrained  method  uses  the  same  logic  as  the 
unconstrained,  but  restrains  accessions  so  that  total  personnel  inventory  does 
not  exceed  approved  end  strength.  This  study  uses  an  accession  plan  for  each 
projected  FY. 

F.  USER  INTERFACE  AND  SCENARIO  SETTINGS 

OSAM’s  graphical  user  interface  (GUI),  named  the  scenario  editor  and 
shown  in  Figure  5,  provides  a  convenient  way  to  build  and  run  a  simulation 
scenario.  Each  scenario  is  a  set  of  input  parameters  and  settings  that  include  a 
user’s  choices  of  preset  input  tables  referred  to  as  plans;  methods  for  applying 
accessions,  promotions,  and  transfers;  and  Structured  Query  Language  (SQL) 
statements  that  are  used  as  options  to  program  any  further  deviations  from  the 
originally  set  parameters  (Mundy,  2014).  The  GUI  allows  users  to  easily  set  input 
parameters,  define  the  modeling  options,  and  specify  desired  output  for  each 
scenario.  The  GUI’s  simulation  building  process  exposes  the  user  to  the  various 
input  parameters  and  modeling  options  via  the  Parameters,  Promotions, 
Accessions,  and  Force-Shaping  tabs,  as  shown  in  Figure  5. 
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Figure  5.  Officer  Strategic  Analysis  Model  Graphical  User  Interface 

1.  Input  Parameter  Settings  and  Modeling  Methods 

The  primary  input  parameters  are  specified  via  the  Parameters  and 
the  Force-shaping  page.  On  the  Parameters  page,  users  can  specify  the  length 
of  the  forecasting  horizon  and  select  from  a  variety  of  input  tables  for 
accessions  plans,  promotion  plans,  Officer  Programmed  Accession  (OPA)  or  end 
strength  plans,  transfer  plans,  and  loss  rates  categories.  The  force-shaping  page 
allows  users  to  add  or  constrain  losses  in  addition  to  the  losses  resulting  from 
the  loss  rates. 

The  Promotion  page,  shown  in  Figure  6,  Accessions  page,  and  Advanced 
page  allow  users  to  choose  among  the  different  modeling  methods  by  which 
OSAM  calculates  or  determines  promotions  and  accessions,  and  transfers, 
respectively.  On  the  promotion  and  accession  page,  users  may  elect  to  use  pre¬ 
existing  plans  in  the  OSAM  database  or  choose  among  the  model’s  modeling 
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methods  to  determine  promotions  and  accessions,  respectively.  The  advanced 
page  allows  users  to  specify  outbound  transfers  by  community  and  paygrade. 
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Figure  6.  Officer  Strategic  Analysis  Model  Promotion  Methods 


2.  Simulation  Model  Output 

Once  each  simulation  scenario  is  completed,  OSAM  consolidates  and 
aggregates  all  output  into  multi-year  summary  (MYS)  tables  and  flow  point  tables 
containing  entity  level  information  for  every  simulation  year.  MYS  tables  produce 
aggregated  forecast  data  from  which  counts  of  beginning  of  year  (BOY) 
inventory,  end  of  year  (EOY)  inventory,  losses,  promotions,  transfers,  and 
accessions  are  generated.  Flow  point  tables  include  detailed  promotion  data  that 
includes  the  number  of  vacancies,  promotions,  and  flow  point  estimates  for  each 
competitive  category  and  control  grades  04,  05,  and  06  (Mundy,  2014).  The 
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OSAM  software  package  also  includes  an  analysis  tool,  built  in  Microsoft  Excel, 
which  is  used  to  analyze  and  track  differences  in  scenario  output. 

The  basic  software  version  of  OSAM  and  the  analysis  tool,  however,  are 
not  suited  for  large-scale  experiments  involving  thousands  of  iterations  as 
needed  for  this  study.  Data  farming  tools  are  built  to  enable  running  multiple 
OSAM  scenarios  without  the  need  for  user  interaction. 

G.  OSAM  SIMULATION  RUNTIME 

The  runtime  for  a  single  iteration  of  an  OSAM  scenario  depends  on  the 
length  of  the  forecasted  horizon  and  the  computing  power  of  the  machine  on 
which  OSAM  is  housed.  A  one  year  forecast  scenario  takes  approximately  two 
minutes  on  a  64-bit  windows  computer  with  an  Intel  2.4  GHZ  processor.  A  seven- 
year  forecast  takes  approximately  eight  minutes  on  the  same  machine. 
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III.  METHODOLOGY  AND  IMPLEMENTATION 


This  chapter  discusses  the  two  primary  methodologies  used  in  this  study, 
Design  of  Experiments  and  Data  Farming.  Design  of  Experiments  (DOE)  is  used 
to  generate  and  explore  a  broad  range  of  OSAM  scenarios.  Data  Farming 
substitutes  OSAM’s  GUI  with  computing  scripts  that  automate  the  process  of 
iteratively  running  and  collecting  data  from  OSAM  scenarios.  Additionally,  the 
chapter  discusses  supplemental  software  tools  used  to  transform  OSAM  into  a 
data  farmable  model. 

A.  DESIGN  OF  EXPERIMENTS 

1.  Definition  and  Applications 

Design  of  Experiments  (DOE)  allows  for  systematically  determining  the 
relationships  between  a  factor,  an  input  affecting  a  process  or  system,  and  the 
response,  the  output  of  that  process  or  system,  using  computational 
experimentation.  Varying  input  factors  singularly,  as  by  haphazardly  choosing 
one  scenario  at  a  time  in  OSAM,  is  time  wasting,  limits  insight,  and  may  produce 
inconclusive  or  even  disastrous  results  (Kleijnen  et  al.,  2005).  With  DOE,  one  can 
carefully  and  efficiently  make  controlled  changes  to  the  input  factors  in  order  to 
gain  maximum  insight  on  how  the  factors  affect  the  outcome  variable.  The  design 
is  specified  as  a  matrix  where  columns  correspond  to  factors  and  rows  to  a 
unique  combination  of  factor  levels,  referred  to  as  design  points  (Kleijnen,  et  al., 
2005).  The  response  surface  generated  from  the  designed  experiment  is  then 
studied  and  analyzed  through  metamodeling  to  gain  insight  on  which,  and  to 
what  extent,  the  simulation  model’s  inputs  affect  the  model’s  output. 

2.  Relevant  Input  Factors 

A  fundamental  initial  step  in  the  design  process  is  identifying  and  selecting 
relevant  input  factors  (Sanchez  et  al.,  2014).  A  factor,  in  the  context  of 
simulation,  is  an  explanatory  variable  to  a  process  or  system’s  output  that  is 
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manipulated  by  the  experimenter.  Decision  factors  are  variables  that  can  be 
controlled  in  the  real  world  by  a  decision  maker  as  opposed  to  noise  factors  that 
happen  randomly,  and  are  therefore  uncontrollable. 

The  choice  of  factors  to  vary  depends  on  the  factors’  characteristics, 
available  computing  resources,  and  the  intent  of  the  underlying  experiment 
(Kleijnen  et  al.,  2005).  This  research  seeks  to  use  OSAM  to  study  the  impact  of 
accession  plans,  and  the  EPOCR  policy  characterized  forced  losses  of  SWO 
officers  with  four,  five,  and  six  years  of  commissioned  service.  Loss  rate 
categories,  transfer  plans,  accession  plans,  and  force-shaping  plans  serve  as  the 
primary  input  variables  of  interest.  Table  1  lists  a  summary  of  the  factors  used  in 
our  experimental  design  and  their  associated  levels.  A  discussion  of  the  factor 
ranges  follows. 


Table  1 .  Experimental  Design  Factors  and  Corresponding  Levels 


Factor 

Variable 

Type 

Levels 

Controllable  Factors 

EPOCR  4  YCS 

Discrete 

[0,  120] 

EPOCR  5  YCS 

Discrete 

[0,  120 

EPOCR  6  YCS 

Discrete 

[0,120] 

Transfer  Plan 

Categorical 

FY[09_10,  14,  15],  No  Transfer 

Accession  plan 

Categorical 

FY[12,  13,  14,  15] 

Uncontrollable  Factors 

Loss  Plans 

Categorical 

FY[06  ,07,  10,  13,  14,  15] 

3.  Factor  Range 

In  order  to  build  an  experimental  design,  it  is  necessary  to  determine  the 
ranges  for  the  relevant  input  factors.  This  section  lists  and  gives  a  general 
description  and  the  range  determination  for  all  the  decision  factors  and  the 
uncontrollable  factor  used  in  this  study. 
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a.  Decision  factors 

This  section  describes  the  decision  factors  used  in  this  study. 

(1)  Transfer  plans 

URL  communities  depend  on  upward  mobility  transfers  and  to  some 
extent  lateral  transfers  to  grow  their  non-accession  inventory.  Transfer  plans  are 
determined  yearly  and  are  effectively  determined  and  controlled  by  strength 
planners  and  policymakers.  OSAM  programs  inbound  and  outbound  transfers  for 
grades  01  to  02,  dimensioned  by  pay  grade,  community,  and  FY  of  execution. 
Grades  01  to  06  transfer  via  training  to  parent  designators  or  via  probationary 
officer  continuation  retention  (POOR)  boards,  whereas  grades  03  to  06  laterally 
transfer  into  other  communities.  For  this  study,  five  transfer  plans  were  chosen 
with  guidance  from  N100  strength  planners  (P.  Saluke,  personal  communication, 
April  26,  2016). 

(2)  Accession  Plans 

Accessions  are  the  primary  method  by  which  communities  get  new 
personnel  inventory.  One  of  the  primary  objectives  of  this  study  is  to  determine 
the  risk  in  future  inventory  posed  by  a  low  accession  plan.  As  with  transfer  plans, 
Navy  policy  makers  determine  and  have  control  over  what  accession  plans  to 
implement,  and  therefore  are  categorized  as  control  factors  for  the  purpose  of 
this  study.  OSAM  programs  accession  using  predetermined  tables  that  represent 
historical  accession  plans  dimensioned  by  designator  and  the  FY  in  which  they 
were  implemented.  For  this  study,  we  selected  four  accession  plans  by  choosing 
the  plans  that  correspond  to  the  most  recent  plan,  the  highest  accession  rate,  the 
lowest  accession  rate,  and  a  moderate  accession  rate. 

(3)  EPOCR  (Four,  Five,  Six  YCS) 

EPOCR  policy  is  simulated  as  user-added  losses.  OSAM’s  flexibility 
allows  users  to  increase  or  decrease  losses  by  community,  pay  grade,  FY  and 
YCS  after  the  natural  losses  are  applied  via  its  force-shaping  functionality.  This 
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thesis  uses  OSAM’s  force-shaping  functionality  to  model  losses  representing 
SWO  officers  referred  to  the  EPOCR  board.  Each  YCS  group  is  represented  as 
an  independent  factor  corresponding  to  SWO  officers  with  four,  five,  and  six 
years  of  commissioned  service.  We  model  EPOCR  losses  with  a  range  of  0  to 
120  losses  for  each  YCS  category.  This  range  represents  the  proposed  total 
number  of  officers  referred  to  the  EPOCR  board. 

b.  Noise  Factors 

This  section  describes  the  uncontrollable  factors  used  in  this  study 

(1)  Natural  Loss  Rates 

Natural  losses  represent  the  uncontrollable  factors  in  manpower  modeling 
caused  by  inherent  uncertainty  in  human  behavior.  For  this  reason,  they  are 
often  the  most  complex  factors  to  model.  OSAM  uses  tabulated  historical  loss 
rates  dimensioned  by  pay  grade,  community,  and  the  FY  or  period  in  which  the 
loss  happens.  (See  Chapter  .1)  This  study  seeks  to  determine  the  variability  in 
OSAM’s  personnel  end  strength  inventory  projection  over  the  uncontrolled 
losses.  The  study  uses  seven  loss  rate  categories  that  represent  a  varied 
continuum  of  years  with  minimal  losses  to  years  with  severe  losses  over  the  past 
ten  years. 

4.  Choice  of  Experimental  Design 

Many  competing  design  exist  in  the  field  of  simulation  analysis.  The  choice 
for  which  design  to  use,  however,  is  significantly  influenced  by  the  number  of 
factors,  type  of  factors,  and  available  computing  resources  (Kleijnen  at  al.,  2005, 
Sanchez  &  Wan,  2012).  This  study  uses  a  combination  of  three  multi-level 
categorical  and  three  discrete  factors  with  a  wide  range  of  values.  The  relatively 
small  number  of  factors,  both  categorical  and,  calls  for  a  mixture  of  a  gridded 
design  and  an  efficient  space-filling  design  applied  to  the  categorical  and  discrete 
variables,  respectively.  Such  a  crossed  design  ensures  all  combinations  of 
categorical  factors  are  captured  and  reveals  interactions  and  other  possible 
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nonlinearities  within  the  results.  A  full  factorial  design  is  appropriate  for  the 
categorical  factors  as  it  will  examine  the  categorical  factors  at  all  levels  and 
guarantee  that  all  main  effects  and  interactions  are  captured  (Sanchez  &  Wan, 
2012).  Nearly  Orthogonal  Latin  Hypercubes  (NOLH)  arise  as  the  most  suitable 
design  for  the  discrete  variables  given  the  broad  range  of  values  for  the  SWO 
forced  losses.  These  designs,  developed  at  the  Naval  Postgraduate  School,  offer 
excellent  space-filling  properties  with  minimal  sampling  (see  Sanchez  et  al., 
2012,  and  Cioppa  &  Lucas,  2007).  They  also  enable  analysts  to  fit  a  broad  range 
of  meta-models  and  generate  a  diverse  set  of  graphs.  For  additional  families  of 
NOLHs,  see  Hernandez  et  al.,  201 2  and  MacCalman  et  al.,  201 6. 

5.  Experimental  Design  Construct 

A  full  factorial  design  for  the  categorical  variables  is  generated  using  the 
Full  Factorial  Design  tool  in  JMP  Pro  12.0.1  (JMP,  2015).  The  three  categorical 
factors  with  six,  four,  and  four  levels,  respectively,  are  crossed  to  get  a  design 
matrix  of  96  design  points.  Figure  7  depicts  the  first  22  of  the  96  design  points  for 
the  full  factorial  design. 
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Figure  7.  Full  Factorial  Design  for  the  Three  Multi-level  Categorical 
Variables  Generated  Using  JMP  Pro  12.0.1 

An  NOLH  design  is  generated  using  Excel  spreadsheet  design  template, 
created  by  Professor  Susan  Sanchez  at  NPS  and  available  at  the  SEED  center’s 
website  https://harvest.nps.edu.  The  particular  design  template  used  generates 
33  design  points  and  can  accommodate  up  to  11  factors,  more  than  sufficient  for 
our  three  discrete  factors.  Figure  8  shows  the  complete  design  matrix  with  the  33 
design  points,  where,  for  instance,  YCS_4  is  a  variable  representing  SWO 
officers  with  four  years  of  commissioned  service  who  have  been  referred  to  the 
EPOCR  board. 
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Figure  8.  NOLH  Design — 3  Factors  and  33  Design  Points 

The  design  point  matrix  for  the  categorical  factors  is  then  crossed  with  the 
design  point  matrix  for  the  discrete  factors  using  JMP.  The  crossing  generates  a 
mixed  design  matrix  with  3,168  design  points.  Figure  9  depicts  a  scatter  plot 
matrix  revealing  the  space  filling  properties  of  the  resulting  crossed  design.  Each 
subplot  within  the  scatter  plot  matrix  shows  a  two  dimensional  projection  of  the 
entire  design  and  the  levels  used  for  each  factor. 
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Scatterplot  Matrix  for  Crossed  Design 
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Figure  9.  Scatter  Plot  Matrix  for  the  Full  Factorial  Categorical  Design 
Crossed  with  the  Discrete  Variable  NOLH  Design 

6.  Replication 

Replication  is  the  repeating  of  the  entire  design  matrix  in  a  simulation 
experiment.  Replication  is  necessary  to  gain  precision  in  its  output  results  and  to 
estimate  the  experimental  error  in  the  design  if  using  a  stochastic  model 
(Cavazzuti,  2013).  However,  a  tradeoff  between  the  number  of  replicates  per 
design  point  and  the  number  of  total  design  points  is  considered  when 

constructing  a  design  (Kleijnen  et  al.,  2005).  Previous  studies  using  the 

stochastically  enhanced  version  of  OSAM  have  shown  the  model  to  depict  very 

little  random  variation  in  end  strength  results.  Borozny  (2015)  used  100 

replications  for  her  experimental  design  and  found  little  stochastic  variation  in 
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OSAM’s  end  strength  results.  DeHollan  (2015)  found  similar  results  using  100 
replications  for  a  single  base  case  design  point.  This  thesis  considers  these 
findings,  as  well  as  conclusions  from  a  base  case  scenario  simulated  for  this 
study,  in  deciding  a  tradeoff  between  replications  and  the  total  number  of  design 
points  to  simulate.  We  use  five  replications  for  each  of  the  3,168  design  points  for 
a  total  of  15,840  simulation  runs. 

B.  DATA  FARMING 

Data  farming,  in  technical  terms,  is  the  process  of  using  rapid  prototyping, 
simulation  modeling,  experimental  design,  high-performance  computing,  and 
data  analysis  to  study  questions  of  interest  with  large  possibility  spaces  (Horne  et 
al.,  2014).  The  process  is  analogous  to  agricultural  farming.  Where  farmers 
manipulate  the  environment  and  cultivate  land  to  maximize  agricultural  yield — 
pest  control,  irrigation,  fertilizer,  etc. — data  farmers  manipulate  simulation  models 
with  designed  experimentation  to  produce  a  landscape  of  output.  A  model  is  data 
farmable  if  it  is  possible  to  programmatically  modify  its  input  and  automatically 
start  an  instance  of  that  model,  usually  with  a  computer’s  command-line  interface 
(Horne  et  al.,  2014). 

OSAM  in  its  original  version  is  not  configured  for  data  farming  as  the 
model  is  incapable  of  running  multiple  scenarios  or  replications  of  single 
scenarios  successively  without  user  interaction.  The  external  software  is 
necessary  to  data  farm  OSAM.  Steve  Upton,  a  Faculty  Associate  for  research  at 
NPS’s  SEED  Center  developed  OSAMFarmer,  a  supplemental  data  farming 
wrapper  that  is  a  collection  of  two  key  software  programs,  OSAMRunner  and 
SimpleFarmer.  Together,  these  programs  “wrap”  around  OSAM  to  make  the 
simulation  model  data  farmable. 

1.  OSAMRunner 

OSAMRunner  strips  out  the  graphical  user  interface  and  allows  a  user  to 

run  successively  one  or  more  replications  of  an  OSAM  scenario  by  feeding  input 

specifications  to  OSAM  via  a  command  line  interface.  The  program  takes  as 

39 


input;  the  model  file;  a  configuration  file  specifying  OSAM  input  parameters  and 
settings  for  a  specific  scenario;  and  a  seed  argument,  where  the  number  of 
seeds  corresponds  to  the  number  of  desired  replications.  The  program 
automatically  saves  the  model’s  output  from  each  completed  replication  and 
successively  starts  the  next  simulation  until  all  replications  are  exhausted.  A  user 
can  then  consolidate  and  analyze  the  model’s  output. 

2.  SimpleFarmer 

The  second  of  the  two  programs,  SimpleFarmer,  concurrently  schedules 
and  manages  a  set  of  predefined  tasks  on  a  single,  multi-processor  machine  by 
distributing  tasks  across  individual  processors.  The  program  takes  as  input  two 
task  files,  one  that  labels  the  names  and  locations  of  OSAM  input  task  elements 
including  the  experimental  design,  and  another  containing  a  set  of  tasks 
corresponding  to  the  simulated  design  points.  SimpleFarmer  schedules  and 
executes  tasks  based  on  predefined  dependencies  and  the  number  of  available 
processers.  Since  SimpleFarmer  distributes  OSAM  runs  across  processors,  one 
can  use  the  software  to  take  advantage  of  parallelized  processing  on  high- 
performance  computer  cluster  nodes  to  cut  down  on  the  model’s  run-time. 

3.  Base  Case 

In  addition  to  the  experimental  design,  this  study  includes  a  base  case 
scenario  to  serve  as  a  baseline  for  comparison  of  the  experimental  results.  The 
base  case  contains  the  same  input  factors  and  parameter  settings  used  for  the 
experimental  design  and  uses  the  most  recent  and  prevailing  manpower  policies 
as  represented  in  OSAM.  No  force-shaping  policies  are  specified  in  the  base 
case  scenario.  Input  parameters  include  the  FY  2016  accession  plan,  FY  2016 
transfer  plan,  and  FY  2015  loss  rates.  The  promotion  method  is  set  to  FY  2017 
promotion  plan  for  FY  2016  and  2017,  and  auto-promote  for  successive  years. 
We  replicate  the  base  case  scenario  100  times  to  further  explore  the  stochastic 
variation  in  end  strength  results,  and  employ  OSAMRunner  to  data  farm  OSAM. 
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C.  EXPERIMENTAL  RUNTIME 

The  runtime  for  a  single  iteration  of  an  OSAM  scenario  depends  on  the 
length  of  the  forecasted  horizon  and  the  computing  power  of  the  machine  on 
which  OSAM  is  housed  and  run.  Consequently,  for  this  study,  the  total 
experimental  runtime  is  the  total  time  it  takes  to  run  all  15,840  of  the  simulation 
runs  specified  in  the  experimental  design,  each  projected  to  seven  years.  A 
seven-year  projection  of  OSAM  takes  approximately  eight  minutes  to  run  using  a 
64-bit  windows  computer  with  an  Intel  2.4  GHZ  processor.  Without  the  benefits  of 
parallelized  cluster  computing,  this  experiment  would  take  approximately  88 
days.  With  the  help  of  NPS’s  SEED  center  resources,  this  study  takes  advantage 
of  OSAMFarmer’s  task  distribution  and  scheduling  properties  to  utilize  a  High- 
Performance  Computing  (HPC)  multi-processor  cluster  system  housed  at  NPS’s 
SEED  center.  The  HPC  reduces  the  total  experimental  time  to  nine  days  (S. 
Upton,  personal  communication,  July  14,  2016). 

This  chapter  explains  and  gives  justification  for  the  methodologies  chosen 
to  explore  the  research  questions  posed  in  Chapter  I.  This  chapter  specifically 
describes  the  approaches  taken  to  generate  simulated  data  from  OSAM.  The 
next  chapter  presents  and  analyzes  the  results. 
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IV.  ANALYSIS  OF  RESULTS 


This  chapter  presents  and  discusses  the  output  from  the  base  case 
scenario  and  the  experimental  design  using  graphical  and  statistical  tools.  We 
analyze  the  base  case  output  to  determine  the  degree  of  stochastic  variation  in 
projected  operating  strength.  Additionally,  we  use  results  from  the  base  case  to 
determine  the  expected  trend  in  URL  inventory  strength  assuming  current  policy 
and  most  recent  loss  rates.  We  then  analyze  experiment  data  to  gain  insight  on 
the  effect  of  accession  plans,  transfer  plans,  and  loss  rate  categories  on 
operating  inventory  strength.  We  further  study  the  impact  of  EPOCR  policy. 
Using  robust  design,  regression  tree  analysis,  and  meta-modeling,  we  determine 
the  policy’s  implications  on  operating  strength  deviation  (OSD). 

A.  ANALYTICAL  TOOLS 

We  use  R  version  3.3.0  (R  Core  Team,  2016)  to  consolidate,  filter,  and 
organize  the  results  from  the  base-case  simulation.  R  is  also  used  for  initial 
exploratory  analysis  of  the  base  case  results.  We  use  JMP  Pro  Version  12.0.1 
(JMP  Pro  2015)  to  process,  organize,  and  filter  the  raw  experimental  output  data 
into  analyzable  form  as  well  as  for  initial  exploratory  analysis.  JMP’s  analytical 
and  graphical  tools  are  used  to  further  explore  the  base  case  and  experimental 
data  for  insight. 

B.  ANALYSIS  OF  BASE-CASE  RESULTS 

Output  from  the  base-case  scenario  is  used  to  assess  the  stochastic 

variation  induced  by  the  modification  of  the  latest  version  of  OSAM  used  for  this 

study.  Additionally,  the  base  case  gives  a  projection  of  the  expected  operating 

inventory  strength  using  standing  policy  and  assuming  the  most  recent  applicable 

loss  rate  category.  This  helps  to  determine  the  expected  direction  of  current 

operating  inventory  assuming  the  assumptions  prevail  over  the  projected  period. 

Specifically,  the  base  case  simulates  100  replications  of  a  single  design  point 

with  FY2016  accession  plan,  FY2016  transfer  plan,  FY2015  loss  Rate,  and 
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FY2017  promotion  plan  for  FY  2016  and  2017,  and  auto-promotion  method  set 
for  successive  years. 


1.  Data  Processing  and  Collection 

OSAMRunner  saves  the  output  from  each  of  the  100  design  points  as  a 
separate  CSV  file.  Each  file  contains  multi-year  summaries,  for  all  70 
designators,  from  which  counts  of  beginning  of  year  (BOY)  inventory,  end  of  year 
(EOY)  inventory,  losses,  promotions,  transfers,  and  accessions  by  designator 
and  pay  grade  are  generated.  We  use  R  to  consolidate  the  CSV  files  and  filter 
the  multi-year  summaries  to  inventory  strength  counts  of  the  specific  URL 
designators  of  interest,  SWO,  SUB,  PILOT,  NFO,  and  their  corresponding 
training  designators.  We  further  group  the  data  by  design  point,  FY,  Grade,  and 
Designator.  Training  designators  are  summed  together  with  their  corresponding 
primary  designators.  For  instance,  SWO  trainee  is  grouped  and  summed  with 
SWO. 


2.  Stochastic  Variation 

To  determine  stochastic  variation  in  the  base  case,  counts  of  inventory 
strength  are  summed  over  all  designators,  grouped  by  FY,  and  averaged  across 
the  100  replications.  The  results  show  that  OSAM  produces  very  little  variation  in 
URL  projected  inventory  strength  results  across  the  100  replications  of  the  base 
case  scenario.  Figures  10  and  11  depict  the  variation  in  the  URL  and  SWO 
inventory  strength,  respectively,  projected  across  the  100  replications  over  a 
seven-year  period.  The  lines  depicted  in  the  figures  correspond  to  different 
replications.  Variance  in  the  total  projected  inventory  increases  in  later  years  in 
both  cases  although  the  variance  is  small  relative  to  the  total  inventory. 
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Figure  10.  URL  Total  Inventory  Strength  Projected  across  100 
Replications  of  Base  Case  Scenario 
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Figure  11.  SWO  Inventory  Strength  Projected  across  100  Replications 

of  Base  Case  Scenario 
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Summary  statistics  for  URL  inventory  for  each  projected  year  are  shown  in 
Table  2.  The  standard  deviation  measures  the  variability  of  the  projected  values 
from  the  mean.  FY  2023  has  the  highest  variation  in  projected  strength  with  a 
standard  deviation  of  12.49  and  a  mean  of  22,890  officers,  suggesting  that 
stochastic  enhancement  of  OSAM  accounts  for  less  than  0.06%  of  the  variation 
in  projected  inventory  strength.  Similar  results  for  SWO  projections  are  shown  in 
Table  3. 


Table  2.  Summary  Statistics  of  URL  Projected  Inventory  Strength  across  100 

Replications  of  Base  Case  Scenario 


FY 

Mean 

Strength 

Std. 

Dev. 

Min 

Max 

25th 

Quantile 

75th 

Quantile 

2016  (BOY) 

24815 

- 

- 

- 

- 

2016 

24958 

2.75 

24952 

24965 

24956 

24960 

2017 

24545 

6.28 

24521 

24556 

24541 

24549 

2018 

24204 

8.57 

24176 

24222 

24199 

24210 

2019 

23911 

10.67 

23884 

23936 

23903 

23918 

2020 

23587 

11.27 

23558 

23618 

23579 

23594 

2021 

23304 

12.42 

23271 

23334 

23295 

23312 

2022 

23073 

12.72 

23023 

23101 

23066 

23081 

2023 

22890 

12.49 

22826 

22909 

22885 

22899 

Table  3.  Summary  Statistics  of  SWO  Projected  Inventory  Strength  across  100 

Replications  of  Base  Case  Scenario 


FY 

Mean 

Strength 

Std. 

Dev. 

Min 

Max 

25th 

Quantile 

75th 

Quantile 

201 6(BOY) 

8365 

- 

- 

- 

- 

- 

2016 

8405 

1.8 

8402 

8409 

8404 

8407 

2017 

8302 

3.84 

8294 

8312 

8300 

8305 

2018 

8229 

5.1 

8215 

8240 

8225 

8232 

2019 

8161 

5.89 

8148 

8178 

8156 

8165 

2020 

8066 

7.03 

8052 

8085 

8061 

8072 

2021 

7993 

8.29 

7967 

8010 

7987 

7999 

2022 

7950 

10.3 

7920 

7976 

7943 

7957 

2023 

7910 

12.41 

7878 

7939 

7902 

7918 
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3.  SWO  Operating  Strength  Deviation-Base  Case 

One  objective  of  this  study  is  to  determine  the  risk  in  operating  strength 
deviation  presented  by  current  policy.  Since  the  base  case  parameters  are  set  to 
reflect  current  manpower  management  policies,  we  use  SWO  inventory  strength 
output  from  the  base  case  to  determine  inventory  risk  by  comparing  projected 
operating  strength  to  planned  Officer  Program  Authorizations  (OPA).  SWO  OPA 
given  by  grade  and  FY  and  programmed  to  FY  2022  are  given  in  Table  4. 

The  mean  of  the  projected  operating  inventory  strength  by  grade  and  FY 
compared  to  planned  authorizations  is  shown  in  Figure  12.  The  total  inventory 
strength  composition  for  the  SWO  community  is  largely  unchanged  for  FY2017 
and  beyond.  Standing  out  is  grade  03  inventory,  which  over-executes 
authorizations  by  more  than  40%  in  all  projected  FYs.  There  is  significant 
increase  in  FY201 7  OPA  for  grade  01 ,  causing  significant  under-execution  in  this 
grade  that  persists  to  the  end  of  the  seven-year  projection  period. 


Table  4.  SWO  Officer  Programmed  Authorization  by  Grade  and  FY 

Source:  N1 00  (2016) 


FY 

ENS 

LTJG 

LT 

LCDR 

CDR 

CAPT 

2016 

1528 

1091 

1388 

1096 

657 

256 

2017 

2289 

1371 

1570 

1082 

651 

243 

2018 

2152 

1379 

1596 

1096 

654 

241 

2019 

2104 

1442 

1608 

1111 

660 

241 

2020 

1993 

1442 

1623 

1103 

661 

238 

2021 

1988 

1441 

1624 

1101 

661 

238 

2022 

1988 

1441 

1624 

1101 

661 

238 
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Figure  12.  Projected  SWO  Inventory  Strength  against  Officer 
Programmed  Authorization  by  Grade  and  FY  (Base  Case) 

C.  INITIAL  ANALYSIS  OF  EXPERIMENTAL  DESIGN  DATA 

This  section  describes  and  assesses  the  experimental  design  data,  with 
EPOCR  policies,  generated  by  the  constructs  of  data  farming  and  design  of 
experiments  described  in  the  previous  chapter.  Whereas  the  experimental  data  is 
used,  in  the  context  of  this  research,  to  assess  the  implications  of  manpower 
policy  plans  and  EPOCR  policies  on  future  Surface  Warfare  inventory  strength, 
the  initial  analysis  assists  in  finding  insightful  information  and  extracting 
elementary  conclusions  from  the  experimental  data  itself.  We  describe  the 
collections  and  processing  of  the  experimental  design  data  and  analyze  the 
effects  of  the  key  input  factors  on  operating  inventory  strength.  Regression  tree 
analysis  is  used  to  determine  the  significant  contributors  to  FY2022  total 
operating  inventory  strength,  for  all  replications  in  the  experimental  design. 


SWO  Projected  Inventory  Strength  vs  Planned  Authorizations  by  Grade 


123456123456123456123456123456123456123456 


FY/ Grade 
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1.  Data  Processing  and  Collection 

Experimental  data  is  consolidated  and  saved  as  Comma-Separated 
Values  (CSV)  file.  The  output  file  contains  results  for  all  the  15,840  design  points, 
with  each  row  in  the  CSV  file  corresponding  to  a  design  point.  The  data  is 
structured  as  a  multi-year  summary  report  containing  all  designators  and  from 
which  counts  of  beginning  of  year  (BOY)  inventory,  end  of  year  (EOY)  inventory, 
losses,  promotions,  transfers,  and  accessions  by  designator  and  pay  grade  are 
generated.  The  file  also  includes  a  column  for  the  random  seed  used  for  each 
design  point,  and  columns  representing  each  of  the  six  factors  in  the 
experimental  design.  A  format  of  the  raw  dataset  produced  from  the  experimental 
design  is  shown  in  Table  5.  Since  the  raw  dataset  is  all-inclusive,  containing 
strength  and  loss  counts  for  all  78  officer  designators,  and  paygrades,  including 
flag  officers  represented  as  grade  07,  we  use  JMP  to  subset  the  raw  data  to 
retain  only  SWO  and  SWO  trainee  inventory  counts  for  grades  01-06,  and 
further  filter  the  count  status  to  inventory  strength  rather  than  loss  counts.  The 
resulting  dataset  is  treated  as  a  parent  dataset  from  which  subsequent  datasets 
are  constructed  for  further  analysis.  The  format  of  the  resulting  parent  dataset 
showing  the  first  10  observations  is  shown  in  Table  6. 


Table  5.  Raw  Data  Format  of  Experimental  Design  Results 


EPOCR  EPOCR  EPOCR 


DP 

LOSS  RATE 

TRANSPUN 

ACC  PUN 

(YCS4) 

(YCSS) 

(YCS  6) 

seed 

FY 

Grade 

Skill 

Status 

EOY  Totals 

1 

1 

FY10 

FV09.11 

FY14 

120 

11 

53 

25841483 

2016 

1 

11  lx  -  Surface  Warfare 

1  Loss  Natural 

2 

2 

1 

FY10 

FY09.11 

FY14 

120 

11 

53 

25841483 

2016 

1 

11  lx-  Surface  Warfare 

4  Loss  -  Force-Out 

1 

3 

1 

FY10 

FY09.11 

FY14 

120 

11 

53 

25841483 

2016 

1 

111x  Surface  Warfare 

5  Loss  User  Ad... 

2 

4 

1 

FY10 

FY09  11 

FY14 

120 

11 

53 

25841483 

2016 

1 

111x  -  Surface  Warfare 

8  No  Change 

167 

5 

1 

FY10 

FY09.11 

FY14 

120 

11 

53 

25841483 

2016 

1 

112x  Submarine  Warfare 

8  No  Change 

35 

6 

1 

FY10 

FY09  11 

FY14 

120 

11 

53 

25841483 

2016 

1 

113x-  Special  Warfare 

8  No  Change 

57 

7 

1 

FY10 

FY09.11 

FY14 

120 

11 

53 

25841483 

2016 

1 

1 14x  -  Special  Operations 

8  No  Change 

7 

8 

1 

FY10 

FY09.11 

FY14 

120 

11 

53 

25841483 

2016 

1 

1 16x  Surface  Warfare  Trainee 

1  Loss  Natural 

33 

9 

1 

fY10 

fY09.11 

FY14 

120 

11 

53 

25841483 

2016 

1 

1 16x  Surface  Warfare  Trainee 

7  Accession 

795 

10 

1 

FY10 

FY09  11 

FY14 

120 

11 

53 

25841483 

2016 

1 

1 1 6x  -  Surface  Warfare  T rainee 

8  No  Change 

683 
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Table  6.  Data  Format  of  the  Filtered  “Parent”  Dataset  Derived  from  the 

Experimental  Design 


▼ 

DP 

LOSS  RATE 

TRANS  PLAN 

ACC  PLAN 

EPOCR 
(YCS  4) 

EPOCR 

(YCS_5) 

EPOCR 

<YCS_6) 

seed 

FY 

Grade 

N 

Rows 

EOY  Totals 

1 

1 

FY10 

FY09_1 1 

FY14 

120 

11 

53 

5732676 

2016 

l] 

3 

1653 

2 

1 

FY10 

FY09_1 1 

FY14 

120 

11 

S3 

5732676 

2016 

2 

4 

1539 

3 

1 

FY10 

FY09  11 

FY14 

120 

11 

53 

5732676 

2016 

3 

4 

2560 

4 

1 

FY10 

FY09  11 

FY14 
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2.  Initial  Assessment  of  Experimental  Results 

Since  data  farming  generates  a  large  amount  of  data,  it  is  worth  first 
exploring  the  output  data  graphically  to  get  an  initial  understanding  of  OSAM’s 
modeling  behavior.  This  section  assesses  OSAM’s  results  by  visualizing  the  total 
inventory  composition  and  measuring  the  effects  of  the  explanatory  input  factors 
on  projected  inventory  strength  for  all  replications  of  the  experimental  design. 

We  start  by  visualizing  the  total  projected  operating  inventory  strength 
profile  across  graders  for  the  projected  period.  Box  plots  of  projected  SWO 
inventory  strength  for  grades  01-06  by  FY  across  all  design  points  are  shown  in 
Figure  13.  The  boxplots  reveal  how  variation  in  inventory  strength  changes  over 
the  projected  FY.  A  smoother  is  added  to  emphasize  the  mean  trend  in  inventory 
strength  by  FY.  For  grades  01  and  02,  variation  increases  in  the  first  few  FYs 
before  declining  and  leveling  off  beyond  2019  and  2021,  respectively.  Grades 
04,  05,  and  06  appear  to  have  a  gradual  increase  in  variation  over  the  projected 
years.  Grade  03  depicts  the  most  variation,  which  also  increases  with  FY.  The 
large  variation  in  this  grade  is  partly  due  to  the  range  of  input  values 
corresponding  to  the  EPOCR  policy.  Grades  01,  02,  and  05  appear  to  have  a 
slightly  negative  trend  while  grade  03  has  a  more  negative  trend  across  all 
projected  FYs. 
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Figure  13.  Box  Plots  of  SWO  Projected  Inventory  Strength  by  Grade 

and  FY  for  Experimental  Design 


Additionally,  of  particular  importance  to  N100  strength  planners,  is  how 
OSAM’s  different  loss  rate  categories  affect  long-term  inventory  strength 
projections.  Box  plots  again  provide  a  manner  in  which  to  compare  and  contrast 
the  variability  in  inventory  strength  results  due  to  each  loss  rate  category.  Box 
plots  in  Figure  14  depict  the  variability  in  mean  total  inventory  strength  for 
FY2023  across  all  design  points.  The  FY2010  loss  rate  appears  to  produce  the 
lowest  mean  in  total  FY2023  projected  inventory  strength  while  FY2006  produces 
the  highest  mean.  The  variation  in  FY2023  projected  total  inventory  strength  is 
lowest  with  the  FY2007  loss  rate  and  highest  with  FY2015  loss  rates.  Appendix  A 
contains  inventory  strength  distribution  plots  and  summary  statistics 
corresponding  to  these  plots. 


51 


8400 


FY  2023  SWO  Total  Inventory  Strength 


Cjn  Strength 


FY06 


FY07 


FY10  FY13 

LOSS.RATE 


FY14 


FY15 


Figure  14.  Box  Plot  of  SWO  Total  Mean  Inventory  Strength  for  FY  2023 

by  Loss  Rates  for  Experimental  Design 


We  further  need  to  determine  whether  the  differences  in  the  mean  total 
inventory  strength  results  due  to  the  different  loss  rate  categories  are  caused  by 
the  effects  of  the  loss  categories  rather  than  random  effects  in  the  modeling 
process.  We  cannot  deduce  such  information  from  the  box  plots  and  a  more 
robust  statistical  test  is  needed.  The  hypothesis  tested  is: 

•  H0  :  There  is  no  difference  in  the  mean  of  total  inventory  strength 
results  in  FY2023  due  to  OSAM’s  loss  rate  categories,  that  is, 

IJi=IJ2= .  / l/6,  where  the  six  populations  refer  to  the  six 

different  OSAM  loss  rate  categories  or  levels  chosen  for  the  study. 

•  Ha\  There  is  a  difference  in  the  means,  that  is,  at  least  two  means 
are  different. 
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We  use  One-way  Analysis  of  Variance  (ANOVA)  to  test  the  hypothesis 
stated  above.  ANOVA  tests  differences  in  means  between  groups  by  partitioning 
the  overall  variance  in  the  response  into  that  due  to  each  of  the  measured  groups 
and  the  errors  (Faraway,  2005).  First,  we  test  the  assumptions  of  ANOVA  to  our 
data  specifically  that  the  mean  inventory  strength  results  across  the  loss  rates 
categories  are  normally  distributed  and  that  the  variance  in  the  results  is  equal 
across  the  loss  rate  categories.  We  see  in  Figure  14  that  these  conditions  are 
generally  met,  that  is:  (1)  all  the  box  plots  depict  a  similar  spread,  suggesting 
equal  variance  among  the  loss  rate  category  results;  and  (2)  all  the  box  plots 
appear  symmetrical  with  centered  interquartile  ranges,  suggesting  normally 
distributed  inventory  strength  results.  An  excerpt  from  the  results  of  the  ANOVA 
test  is  depicted  in  Figure  15.  The  y-axis  is  the  mean  total  inventory  strength 
results  across  all  design  points.  The  vertical  span  of  the  diamond  shapes 
represents  a  95%  confidence  interval  around  the  mean  of  each  loss  rate 
category.  The  mean  is  represented  by  the  center  vertical  line  in  each  diamond. 
The  size  of  the  intersection  of  the  circles  measures  the  statistical  difference  in  the 
mean  total  inventory  strength  results  among  each  paired  loss  rate  category, 
where  the  larger  the  intersection,  the  smaller  the  difference  in  means.  Visual 
inspection  of  the  ANOVA  test  reveals  significant  differences  among  the  means. 
Statistical  results  shown  in  Figure  16  reveal  a  significant  F-statistic,  p-value  < 
0.001,  which  supports  the  visual  conclusion  that  there  are  significant  differences 
in  the  inventory  strength  results  produced  by  the  loss  rates. 
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Figure  15.  One-way  Analysis  of  Mean  Variance  of  FY2023  Total 

Inventory  Strength  by  Loss  Rate 


FY06 


FY07 


FY10  FY13 

LOSS  RATE 


FY14 


FY15 


All  Pairs 

Tukey-Kramer 

0.05 


Analysis  of  Variance 

Source 

Sum  of 
DF  Squares 

Mean  Square 

F  Ratio 

Prob  >  F 

LOSS.RATE 

5  25488285 

5097657 

34.2947 

<.0001‘ 

Error 

3162  470007562 

148642 

C.  Total 

3167  495495847 

Figure  1 6.  ANOVA  Statistical  Test  for  Loss  Rates 

Similar  ANOVA  hypotheses  are  tested  for  transfer  plans  and  accession 
plans  to  determine  the  variables’  respective  effects  on  projected  inventory 
strength.  One  interesting  observation  from  Figure  17  is  that  having  no  transfers 
produces  a  significantly  higher  projected  inventory  strength  value  than  when 
using  a  transfer  plan.  One-way  ANOVA  results  by  accession  plans  are  shown  in 
Figure  18.  Projected  inventory  strength  values  for  FY  2015  and  2014  accession 
plans  are  statistically  indistinguishable  and  have  the  lowest  mean.  FY2012 
accession  plan  has  the  lowest  projected  inventory  strength  value,  which  is  also 
statistically  different  from  the  values  from  the  rest  of  the  accession  plans. 
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Figure  17.  One-Way  Analysis  of  Mean-Variance  of  FY2023  Total 

Inventory  Strength  by  Transfer  Plan 
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Figure  18.  One-Way  Analysis  of  Mean-Variance  of  FY2023  Total 
Inventory  Strength  by  Accession  Plan 
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3.  EPOCR  Policy 

We  continue  our  initial  analysis  by  determining  the  effects  of  implementing 
an  EPOCR  policy  to  operating  inventory  strength  projections.  For  this,  we 
summarize  the  mean  inventory  strength  results  across  the  design  space  by 
grade  and  FY  and  compare  the  results  to  Officer  Programmed  Authorizations 
(OPA).  A  composition  of  SWO  inventory  strength  by  grade  and  FY  with  Projected 
inventory  strength  compared  against  OPA  is  shown  in  Figure  19.  We  observe 
over-execution  persisting  in  grade  03  in  FY2016-2019,  and  gradually  decreasing 
to  healthy  levels  by  FY  2022.  FY2022  inventory  strength  summary  statistics  for 
each  pay  grade  is  shown  in  Table  7. 
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Figure  19.  Projected  SWO  Inventory  Strength  with  EPOCR  Policy 
Against  Officer  Programmed  Authorization  by  Grade  and  FY 
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Table  7.  Summary  Statistics  of  Mean  SWO  FY  2022  Projected  Inventory 
Inventory  Strength  Composition  Using  EPOCR  Policy 


Figures  20  and  21  show  projected  inventory  strength  profiles  for  the 
lowest  (FY2012)  and  highest  (FY2015)  accession  plan,  respectively.  An 
interesting  thing  to  note  is  that  there  is  minimal  difference  between  the  low  and 
high  accession  plan  inventory  projection  profiles  for  grades  02  and  03, 
confirming  our  conclusions  from  the  one-way  ANOVA  results  for  the  accession 
plans. 
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Figure  20.  Projected  SWO  Inventory  Strength  with  EPOCR 

Using  a  Low  Accession  Plan 

57 


^Inventory  Strength  versus  Authorizations  (EPOCR-High  Accession  Plan) 
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Figure  21.  Projected  SWO  Inventory  Strength  with  EPOCR 

Using  a  High  Accession  Plan 


4.  Regression  Tree  Analysis 

Regression  trees  offer  a  result  structure  that  is  more  intuitive  and  are  less 
computationally  tasking  than  linear  or  polynomial  regression.  Additionally,  they 
are  powerful  at  identifying  influential  factors  affecting  an  outcome  variable,  as 
well  as  finding  interactions  among  variables  (Faraway,  2005).  The  general 
regression  tree  methodology  also  allows  analysis  for  a  combination  of  discrete 
and  categorical  predictor  variables  as  used  in  this  study. 

We  use  regression  tree  analysis  to  determine  the  most  influential  factors 
of  total  SWO  inventory  strength  for  grades  01  to  06  in  FY2022.  The  results  of 
the  regression  tree  model  for  total  inventory  strength  are  shown  in  Figure  22.  The 
first  node,  the  parent  leaf,  or  root  leaf,  shows  the  mean  and  standard  deviation  of 
the  total  inventory  strength  in  the  entire  dataset,  7,375  and  365,  respectively,  and 
the  total  count  of  all  observations,  3,168.  The  data  is  recursively  partitioned  into 
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regions  corresponding  to  the  distinct  paths  from  the  parent  leaf  to  the  leaves  with 
no  further  splits,  the  terminal  leaves.  The  intermediate  cells  between  the  parent 
and  terminal  leaves  are  child  leaves.  A  predictive  model  is  fitted  for  each  leaf  and 
a  corresponding  estimate  of  the  mean  total  inventory  strength  is  determined  for 
that  leaf.  The  title  cell  shows  the  number  of  splits  and  the  R-square  value,  which 
describes  how  well  the  tree  explains  the  variance  in  mean  total  projected 
inventory  strength  results.  The  most  influential  factors  in  determining  the 
projected  mean  total  inventory  strength  in  order  of  importance  in  the  tree  are 
transfer  plans,  EPOCR  policy  with  4  YCS,  and  EPOCR  with  5  YCS.  The  R- 
Square  value  of  0.733  suggests  that  the  tree  model  explains  about  73%  of  the 
variation  in  total  end  strength  results.  An  expected  observation,  as  revealed  in 
the  first  split  of  the  tree,  is  that  having  no  transfer  plan  significantly  increases  the 
total  mean  inventory  strength.  Each  distinct  path  from  the  terminal  to  the  parent 
leaf  also  represents  a  sequence  of  interacting  terms.  Using  a  transfer  plan  with 
an  EPOCR  policy  that  refers  64  or  more  officers  with  four  YCS,  and  53  or  more 
officers  with  5  YCS  will  results  in  the  least  mean  total  inventory  strength.  On  the 
contrary,  having  no  transfers  and  referring  fewer  than  64  officers  with  four  YCS 
results  in  the  largest  projected  inventory  strength  value. 
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Figure  22.  Regression  Tree  for  SWO  Total  Inventory  Strength 

in  FY2022. 

D.  ANALYSIS  OF  OPERATING  STRENGTH  DEVIATION 

Of  importance  to  this  research,  and  of  interest  to  N100,  is  determining  the 
best  manpower  policies  that  will  minimize  the  deviation  of  projected  SWO 
inventory  strength  from  planned  authorizations  at  a  given  future  period.  We  will 
specifically  focus  on  the  total  FY2022  projected  inventory  strength  for  grades  03 
to  06.  We  have  already  determined  that  there  is  significant  variation  in  mean 
total  inventory  strength  across  the  different  manpower  policies  used  in  this  study. 
Furthermore,  we  have  identified  the  most  influential  factors  that  affect  projected 
total  inventory  strength.  This  section  uses  tree  analysis  to  find  the  most  influential 
factors  that  determine  FY2022  total  OSD  for  grades  03  to  06.  Concepts  of 
robust  design  and  meta-modeling  are  used  to  determine  manpower  policy 
settings  that  minimize  projected  total  OSD  in  these  grades. 
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1.  Regression  Tree  Total  Operating  Strength  Deviation 

We  use  regression  tree  analysis  to  identify  influential  factors  determining 
total  operating  strength  deviation  (OSD)  over  the  loss  rates  for  grades  03  to 
grade  06.  To  set  up  our  dataset  for  this  analysis,  we  subset  the  parent  dataset  to 
grades  03  to  06  and  FY2022  and  introduce  a  column  in  the  resulting  dataset 
containing  the  OSD  at  each  design  point.  We  then  sum  OSD  over  the  grades  by 
all  the  input  factors  and  random  seeds.  The  resulting  format  of  the  data  with  a 
column  for  the  sum  of  OSD  represented  as  sum  (Deviation)  is  shown  in  Table  8. 
We  use  JMP  to  build  a  regression  tree  model  on  the  mean  total  OSD.  The 
resulting  regression  tree  is  shown  in  Figure  23.  The  variables  in  the  regression 
tree  model  at  six  splits  explain  about  63%  of  the  variation  in  OSD.  Transfer  plan, 
EPOCR  5  YCS,  and,  EPOCR  4  YCS  are  most  significant  in  determining  the 
amount  of  OSD. 


Table  8.  Data  Format  with  Inventory  Strength  Deviation  Column 


LOSS  RATE 

TRANS  PLAN 

ACC  PLAN 

EPOCR 
(YCS  4) 

EPOCR 
(YCS  5) 

EPOCR 
(YCS  6) 

seed 

N 

Rows 

Sum 

(Deviation) 

1 

FY06 

FY09J0 

FY12 

0 

109 

68 

14143576 

4 

482 

2 

FY06 

FY09J0 

FY12 

0 

109 

68 

35989036 

4 

531 

3 

FY06 

FY09_10 

FY12 

0 

109 

68 

46807864 

4 

528 

4 

FY06 

FY09_10 

FY12 

0 

109 

68 

56621581 

4 

518 

5 

FY06 

FY09_10 

FY12 

0 

109 

68 

80735315 

4 

518 

6 

FY06 

FY09J0 

FY12 

4 

8 

83 

255496 

4 

864 

7 

FY06 

FY09_10 

FY12 

4 

8 

83 

29743319 

4 

850 

8 

FY06 

FY09_10 

FY12 

4 

8 

83 

31222864 

4 

821 

9 

FY06 

FY09_10 

FY12 

4 

8 

83 

68509862 

4 

840 

10 

FY06 

FY09J0 

FY12 

4 

8 

83 

73352979 

4 

838 
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Figure  23.  Regression  Tree  of  Mean  total  Operating  Strength  Deviation 

for  grades  03  to  06 


2.  Robust  Design  and  Construct 

Robust  design  is  a  powerful  system  optimization  technique  that  takes  the 
approach  of  finding  ways  to  determine,  with  consistency,  accurate  and  precise 
response  values  by  controlling  for  uncertainty  in  the  system.  Both  the  mean  and 
variation  of  the  response  are  considered,  where  a  system  is  optimized  by 
choosing  the  closest  mean  to  a  target  value  with  the  smallest  variance  (Sanchez, 
2000).  The  idea  is  to  select  alternatives  that  best  approximate  the  desired 
threshold  value  for  their  mean,  as  well  as  produce  a  reasonably  small  variance 


62 


around  that  threshold.  Variability  in  robust  analysis,  then,  represents  a  risk  or 
cost  in  the  system  modeled. 

The  approach  in  robust  analysis  is  to  separate  the  explanatory  factors  of  a 
response  variable  into  controllable  and  uncontrollable  factors  and  then  use  a  loss 
function  to  find  the  best  configuration  of  the  explanatory  factors  that  produces  the 
best  results  with  minimal  variability  across  uncontrollable  factors.  The  response 
variable  of  interest  is  the  operating  strength  deviation  and  the  controllable 
variables  are  represented  by  the  manpower  plans:  accession  plan,  the  transfer 
plan,  and  the  three  EPOCR  variables.  Loss  rate  categories  represent 
uncontrollable  factors  corresponding  to  the  uncertainty  in  the  real  world.  Loss  or 
risk  is  a  function  of  the  variance  and  mean  of  the  operating  strength  deviation, 
and  is  given  by  the  formula 

E(Loss)  =  variance(deviatiori)  +  (mean(deviation))2  ^  ^  ^ 

where  E(Loss)  is  the  long  run  cost  or  risk  of  operating  strength  deviation,  and 
where  zero  is  the  best  possible  value  of  the  deviation.  The  loss  function 
penalizes  for  both  a  higher  variance  and  for  the  mean  straying  away  from  its  best 
possible  value.  In  other  words,  if  both  values  are  low,  the  manpower 
configuration  is  robust  and  should  perform  consistently  well  in  minimizing 
operating  strength  deviation.  The  second  term  is  squared  to  level  the  units  for 
both  terms  to  units-squared. 

To  set  up  our  data  for  robust  analysis,  we  further  process  Table  8  data  by 
averaging  the  mean  and  variance  of  the  summed  deviation  over  the  loss  rates. 
The  resulting  dataset  is  depicted  in  Table  9.  We  then  proceed  to  build  a  meta¬ 
model  on  operating  strength  deviation. 
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Table  9.  Data  Format  of  Robust  Design  Dataset  with  Loss  Column. 


B 

TRANS  PLAN 

ACCLPLAN 

EPOCR 
(YCS  4) 

EPOCR 

(YCS_5) 

EPOCR 
(YCS  6) 

N  Rows 

Mean 

(Sum(Deviation)) 

Variance 
(Sum  (Deviation)) 

1 

FY09J0 

FY1 2 

0 

109 

68 

30 

655.567 

30437.426 

2 

FY09_10 

FY12 

4 

8 

83 

30 

1011.200 

36346.579 

3 

FY09_10 

FY12 

8 

116 

64 

30 

615.967 

30008.171 

4 

FY09J0 

FY12 

11 

0 

105 

30 

925.000 

34487.655 

5 

FY09_10 

FY1 2 

15 

68 

11 

30 

1032.733 

34342.961 

6 

FY09J0 

FY12 

19 

49 

75 

30 

816.300 

32853.734 

7 

FY09_10 

FY1 2 

23 

45 

49 

30 

934.000 

34426.897 

8 

FY09_10 

FY12 

26 

34 

41 

30 

1006.100 

34183.197 

9 

FY09J0 

FY12 

30 

41 

86 

30 

760.067 

32377.789 

10 

FY09_10 

FY12 

34 

94 

30 

30 

766.333 

31219.126 

3.  Meta-Models  for  Mean  and  Variance  of  Operating  Strength 
Deviation 

We  extend  our  analysis  to  build  a  meta-model  that  determines  robust 
parameters  for  correcting  FY2022  total  operating  strength  deviation  for  grades 
03-06.  Meta-models  act  as  proxies  to  the  entire  simulation  model  by  offering  a 
more  computationally  efficient  way  of  computing  response  estimates  from  the 
simulation  model  (Law,  2000).  We  use  meta-modeling  to  understand  the 
relationship  between  the  controllable  input  factors’  settings  for  the  accession 
plans,  transfer  plan,  and  the  three  EPOCR  variables  with  the  response  variables 
represented  by  the  mean  and  variance  of  OSD.  The  meta-model  quantifies  the 
effects  of  each  of  the  controllable  variables  on  both  the  mean  and  variance  of 
operating  strength  deviation. 

Using  JMP,  we  simultaneously  fit  meta-models  on  the  mean  and  variance 
of  operating  strength  deviation  using  stepwise  regression  and  with  minimum 
Bayesian  information  criterion  (BIC)  selected  as  the  stopping  condition  for  fitting 
the  best  model.  We  screen  out  less  significant  factors  from  the  resulting  models 
and  use  the  remaining  factors  to  fit  a  model  using  all  two-way  interactions  and 
second-degree  polynomials  terms.  We  further  filter  out  insignificant  factors  from 
the  resulting  models,  leaving  only  statistically  significant  factors  and  interactions. 
We  use  data  splitting  to  avoid  over-fitting  and  to  ensure  that  the  models  have 
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significant  prediction  power.  For  this,  the  dataset  used  to  build  the  model,  Table 
9,  is  randomly  partitioned  into  a  training  set  that  is  75%  of  the  data,  and  is  used 
to  build  the  models.  The  rest  of  the  data  is  set  aside  as  a  test  set  and  is  used  to 
validate  the  predictive  power  of  the  model. 

A  screen  shot  of  summary  statistics  and  diagnostic  plots  for  the  resulting 
meta-model  built  with  the  training  set  is  depicted  in  Figure  24.  The  summary 
statistics  indicate  good  predictive  capability.  All  five  input  factors  are  statistically 
significant  with  t-statistic  p-values  <  0.01.  Additionally,  the  model  has  a  relatively 
high  R-Square  value  of  0.999.  The  residuals  are  also  sufficiently  normal  and 
exhibit  constant  variance,  as  evidenced  by  the  Normal  Quantile  and  Residual  by 
Predicted  plots,  suggesting  that  the  model  has  a  good  fit  on  the  data.  A  bivariate 
fit  of  predicted  values  from  the  training  and  test  set  is  shown  in  Figure  25.  The 
linear  diagonal  fit  indicates  that  the  model  has  good  predictive  performance. 
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4  Summary  of  Fit 

RSquare 

0.999951 

RSquare  Adj 

0.99995 

Root  Mean  Square  Error 

2.577441 

Mean  of  Response 

724.2183 

Observations  (or  Sum  Wgts) 

394 

4  Effect  Tests 

Source 

Nparm 

DF 

Sum  of 
Squares 

F  Ratio 

Prob  >  F 

TRANS_PLAN 

3 

3 

25392517 

1274111 

<.0001* 

ACC.PLAN 

3 

3 

438562 

22005.57 

<.0001* 

SWO.YCS.4 

1 

1 

6636151 

998938.8 

<.0001* 

SWO.YCS.5 

1 

1 

9553868 

1438142 

<.0001* 

SWO.YCS.6 

1 

1 

9559031 

1438919 

<.0001* 

4  Residual  by  Predicted  Plot 


0  200  400  600  800  1000  1200  14001600 
Mean  (Sum(Deviation))  Predicted 


~  Residual  Mean  (Sum(Deviation)) 


-8  -6  -4  -2  0  2  4  6  8 


Figure  24.  Diagnostic  Plots  for  Mean  Total  End  Strength  Deviation 

Meta-Model  Strength 
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Normal  Quantile  Plot 


0  200  400  600  800  1000  1200  1400  1600 

Pred  Formula  Sum(Mean(Deviation))  2 


Figure  25.  Actual  versus  Predicted  Plot  for  Test  Set  Using 
Mean  Total  Inventory  Strength  Meta-model 

4.  Prediction  Profiler 

JMP’s  prediction  profiler  offers  an  interactive  way  of  exploring  a  model’s 
response  surface  by  varying  the  input  factors.  The  profiler  displays  the  predicted 
mean  and  variance  of  operating  strength  deviation  as  one  input  variable  is 
changed  while  holding  the  others  constant.  We  can  use  the  prediction  profiler  to 
find  the  robust  settings  that  minimize  both  the  mean  and  variance  of  the  OSD 
variable.  A  prediction  profiler  with  function  curves  for  the  mean  and  variance  of 
deviation  is  shown  in  Figure  26.  The  transfer  plan  and  accession  plan  are  set  to 
FY2015  and  EPOCR  parameters  are  set  to  zero.  This  corresponds  to  the  status 
quo  where  no  EPOCR  policy  is  implemented.  Shifting  the  input  parameters  on 
the  x-axis  along  the  deviation  curves  results  in  a  marginal  change  in  deviation 
proportional  to  the  slope  of  the  curve.  Thus,  a  curve  with  a  slope  of  zero  will  have 
no  effect  on  the  response.  The  response  at  these  values  indicates  a  mean  OSD 
of  1261  with  a  standard  deviation  of  189,  suggesting  that  total  projected 
operating  inventory  strength  in  grade  03  to  06  will  over-execute  OPA  by  an 
estimated  1261  officers  in  FY2022  where  an  EPOCR  plan  is  not  implemented. 
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This  again  assumes  that  that  Navy  implements  the  respective  manpower  policies 
each  year  over  the  projected  period.  Shifting  the  transfer  plan  to  no  transfers 
increases  the  mean  OSD,  as  does  shifting  the  EPOCR  value  setting  to  the  right. 


Prediction  Profiler 


TRANS.PLAN 


ACC.PLAN 


(YCS_4) 


(YCS_5) 


(YCS_6) 


Figure  26.  Prediction  Profiler  of  Mean-Deviation  Meta-Model 


Figure  27  depicts  a  prediction  profiler  with  transfer  and  accession  plans 
set  at  their  respective  values  from  the  robust  analysis.  The  EPOCR  parameters 
are  arbitrarily  set  to  a  combination  such  that  OSD  is  reduced  to  a  modest  level  of 
300  with  a  standard  deviation  of  141.  This  suggests  that  the  total  inventory  for 
grades  03  to  06  will  over-execute  OPA  by  a  mean  estimate  of  300  officers  in 
FY2022  assuming  this  policy  prevails  over  the  projected  period. 
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’  Prediction  Profiler 


Z  90.9  90.9  37.8 

FY14  FY12  EPOCR  EPOCR  EPOCR 

TRANS.PLAN  ACC.PLAN  (YCS_4)  (YCS_5)  (YCS_6) 


Figure  27.  Prediction  Profiler  of  Mean-Deviation  Meta-Model  With 
Robust  Settings  for  Transfer  and  Accession  Plans. 


Figure  28  depicts  a  prediction  profiler  with  robust  settings  for  transfer  and 
accession  plans  with  EPOCR  parameters  arbitrarily  set  to  a  combination  that 
reduces  OSD  to  a  mean  estimated  value  of  just  two  officers  in  FY2022.  The 
standard  deviation  for  this  setting  is  123  officers,  suggesting  an  enhanced  risk  of 
under-executing  SWO  inventory. 


69 


Variance  Mean 

(Sum(Deviation))  (Sum(Deviation)) 


w  Prediction  Profiler 
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Figure  28.  Prediction  Profiler  with  Robust  Settings  Converging 
Operating  Strength  Deviation  to  two  Officers 
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V.  CONCLUSION  AND  RECOMMENDATIONS 


This  thesis  extends  the  works  of  Sibley  (2012),  Borozny  (2015),  and 
DeHollan  (2015)  to  enhance  OSAM’s  utility  as  an  inventory  projection  model 
using  data  farming  and  design  of  experiments.  The  insights  gained  from  this 
study  go  beyond  the  scope  of  the  specific  questions  guiding  this  research.  To 
illustrate  OSAM’s  utility,  this  thesis,  first,  analyzes  the  effects  of  current 
manpower  policy  on  the  future  composition  of  URL  inventory.  Second,  this  thesis 
analyzes  the  effects  of  historical  accession  plans  and  the  proposed  EPOCR 
policy  on  surface  warfare  officer  inventory.  Finally,  the  study  identifies  the  risk 
inherent  in  the  EPOCR  policy  by  measuring  projected  inventory  deviation  from 
planned  authorizations  and  identifies  policies  that  mitigate  identified  risk.  The 
section  that  follows  answers  the  specific  research  questions  posed  for  this  study 

A.  ANSWERS  TO  RESEARCH  QUESTIONS 

1.  What  Is  the  Expected  Short-  and  Long-Term  Trend  in  URL 

Operating  Inventory  Assuming  Current  Policy  and  Prevailing 
Assumptions? 

OSAM  models  inventory  projections  dynamically  over  annual  time-steps, 
and  as  such,  provides  the  ability  to  track  URL  inventory  over  the  entire  projected 
period.  The  base-case  projects  2016  URL  inventory  over  a  seven-year  period 
with  a  key  assumption  that  the  current  policy,  FY2015  accessions,  transfer  plans, 
and  loss  rates  prevail  over  the  projected  period.  We  find  that  total  operating 
inventory  strength  falls  by  an  average  of  8%  over  the  seven-year  period  from 
24,815  officers  at  the  beginning  of  2016,  to  a  mean  and  standard  deviation  of 
22,890  and  13  officers  at  the  end  of  2023.  The  mean  Inventory  strength  results  in 
the  base  case  have  extremely  small  standard  deviation  values,  suggesting  little 
value  in  using  OSAM  as  a  stochastic  model. 
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2.  What  Are  the  Long-Term  Risks  in  Inventory  for  the  SWO 
Community  Associated  with  Current  Policy? 

We  measure  risk  as  operating  strength  deviation,  the  magnitude  of 
expected  deviation  of  operating  inventory  from  planned  authorizations  (OPA). 
The  base-case  results  show  over  40%  over-execution  in  grade  03  that  is 
expected  to  persist  over  the  entire  projected  period.  We  should  remind  the  reader 
that  the  base-case  assumes  current  manpower  policy  and  loss  rates  prevailing 
over  the  projected  period.  In  addition,  the  current  accession  plan  is  not  enough  to 
sustain  an  increase  in  programmed  authorizations  for  grade  01  planned  for 
FY2017  and  beyond.  Interestingly,  the  base  case  shows  healthy  inventory 
strength  for  grades  04,  05,  and  06. 

3.  What  Is  the  Impact  of  the  Enhanced  Probationary  Officer 
Continuation  and  Re-designation  Board  on  SWO  Inventory? 

The  EPOCR  results  assume  averages  over  all  practical  manpower 
policies  as  used  in  this  study.  Additionally,  the  results  rest  on  the  assumption  that 
the  policies  prevail  over  the  projected  period.  We  find  that  implementing  the 
policy  significantly  reduces  grade  03  over-execution  by  FY2022,  and  levels  the 
average  operating  strength  in  all  grades  to  match  OPA.  Total  operating  inventory 
strength  is  reduced  by  12%  over  the  seven-year  period,  from  8,365  officers  at  the 
beginning  of  FY2016  to  a  mean  and  standard  deviation  of  7,375  and  739  officers 
at  the  end  of  FY2023.  Total  SWO  over-execution  is  reduced  to  a  modest  4%  over 
FY2022  OPA  from  34%  in  FY2015.  The  extremely  high  standard  deviation  on 
FY2022  total  SWO  inventory  suggests  a  high  risk  of  under-executing.  Whereas 
the  EPOCR  policy  will  generally  have  a  correcting  impact  on  operating  strength 
deviation,  caution  should  be  taken  on  how  to  implement  it  to  avoid  under¬ 
execution.  Robust  analysis  of  the  policy  can  identify  how  to  best  implement 
EPOCR. 

Focusing  on  the  total  inventory  of  grades  03  to  06,  we  use  robust 
analysis  and  meta-modeling  to  determine  the  risks  in  implementing  EPOCR,  and 
find  the  best  manpower  policies  that  will  correct  operating  strength  deviation  in 
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these  grades.  Our  initial  finding  suggests  that  using  a  transfer  plan  and 
implementing  an  EPOCR  policy  for  surface  warfare  officers  with  4  and  5  YCS 
had  the  most  significant  effect  on  total  OSD.  We  find  that  a  low  accession  plan, 
that  of  FY2012,  and  a  high  transfer  plan,  that  of  FY2014,  are  the  most  robust 
non-EPOCR  policies  to  correct  OSD.  All  accession  plans  used  in  the  study  had 
relatively  low  impact  on  OSD,  suggesting  that  an  economically  efficient  plan,  a 
low  accession  plan,  holds  the  same  risk  as  a  high  accession  plan.  Additionally, 
the  FY2014  transfer  plan  produced  the  lowest  OSD.  Prediction  profilers  on  meta¬ 
models  of  OSD  provide  evidence  that  EPOCR  can  effectively  correct  operating 
end-strength  to  OPA. 

The  study  shows  that  it  is  possible  to  meet  future  SWO  manpower 
requirements  with  a  conservative  accession  plan  coupled  with  an  aggressive 
transfer  plan  referring  officers  to  other  communities.  The  Navy  should  additionally 
supplement  its  current  force-shaping  tools  with  EPOCR  to  manage  officer  over¬ 
execution.  When  used  correctly,  EPOCR  has  a  potential  to  minimize  over¬ 
execution  in  the  SWO  community  with  minimal  and  measurable  risk  of  under¬ 
executing.  The  number  of  officers  within  each  YCS  group  refereed  to  EPOCR 
should  be  selected  in  such  a  manner  to  account  for  other  constraints, 
assumptions,  and  interests  external  to  the  scope  of  this  research. 

B.  RECOMMENDATIONS  FOR  FUTURE  STUDIES 

This  study  demonstrates  the  utility  of  OSAM  as  an  officer  projection  model 
by  applying  efficient  methods  of  extracting  information  from  the  model  and 
providing  a  quantitative  framework  to  analyze  the  model’s  results.  There  exists 
potential  for  research  that  builds  on  this  study  or  improves  on  OSAM’s  utility. 

1.  Continuous  Input  Levels 

Future  work  on  OSAM  can  effectively  enrich  the  factor  space  in  the 

experimental  design  by  varying  beginning  of  year  inventory,  loss  rates,  transfer 

plans,  and  accession  plans  as  continuous  rather  than  categorical  variables.  The 

current  version  of  OSAM,  and  as  used  by  this  study,  represents  these  input 
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variables  as  a  set  of  pre-established  tables  corresponding  to  historical 
realizations  of  these  plans,  given  by  FY,  designator,  and  paygrade.  Since  OSAM 
retrieves  the  inputs  as  numerical  sets,  loss  rates,  transfer  plans,  and  accessions 
are  modeled  as  categorical  variables  in  the  experimental  design  construct,  where 
the  factors  in  the  experimental  design  correspond  to  the  manpower  plans,  and 
the  factor  levels  to  a  numeric  constant  for  each  designator  and  grade.  The  factor 
levels  do  not  cover  a  realistic  range  of  possibilities  for  the  manpower  plans. 
Additionally,  we  cannot  prescribe  numeric  values  for  the  manpower  plans  in  our 
results.  However,  using  OSAM’s  database  structure,  we  can  use  an  adjustment 
factor  to  effectively  update  input  values  for  the  plan  parameters  in  the 
experimental  design  through  SQL  statements.  For  example, 


UPDATE  AccPlan_FY16  SET  Accessions=Accessions*1 .5  WHERE 
‘Community’=SWO  TRAINEE 


will  increase  FY2016  SWO  accessions  by  a  factor  of  1.5.  BOY  inventory,  loss 
rates,  and  transfer  plan  inputs  can  be  adjusted  in  a  similar  way.  We  can  then 
vary  the  adjustment  factors  in  the  experimental  design  to  construct  a  realistic 
range  of  numeric  values  that  cover  the  manpower  plans.  An  experimental  design 
varying  the  manpower  plans  as  discrete  factors  would  give  a  more  precise 
assessment  as  to  how  the  numeric  levels  of  the  manpower  plans  affect  projected 
inventory  strength. 

2.  Fiscal  Year  Fixed-Effects 

This  thesis  also  uses  linear  projections  of  OSAM  inventory  over  the 
projected  window.  The  manpower  and  force-shaping  plans  (EPOCR)  are 
modeled  with  the  assumptions  that  the  specific  input  parameters  for  each 
scenario  are  implemented  linearly  and  annually  over  the  projected  window. 
Further  research  should  consider  projected  year  fixed-effects  for  each  of  the 
manpower  and  force-shaping  plans.  The  advantage  of  such  a  construct  is  that 
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the  research  can  identify  an  optimal  manpower  policy  for  each  projected  fiscal 
year.  We  will  add  to  this  note  that  adding  fixed-year  effects  will  compound  the 
number  of  factors  needed  by  a  factor  proportional  to  the  number  of  projected 
years.  However,  the  computational  resources  needed  to  account  for  additional 
factors  are  minimized  by  using  an  appropriate  experimental  design.  Additionally, 
since  OSAM  has  very  little  variation,  replication  adds  little  value  to  an 
experiment.  Construction  of  any  designed  experiment  for  future  work  should  be 
weighted  in  favor  of  more  design  points  rather  than  number  of  replications. 
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End  Strength 


APPENDIX  A.  SWO  PROJECTED  INVENTORY  BY  GRADE  AND 
FY  ACROSS  DESIGN  POINTS-FY2023. 


01  End  Strength 


02  End  Strength 


— i - r 

2022 


03  End  Strength 


04  End  Strength 


End  Strength 


APPENDIX  B.  DISTRIBUTION  OF  PROJECTED  END  STRENGTH 
BY  LOSS  RATE  CATEGORY-FY2023. 


^  -  Distributions  LOSS_RATE=FY10 
-  Sum(Mean(Strength)) 


6200  6500  6800  7100  7400  7700  8000 


Summary  Statistics 

Mean 

7150.1481 

Std  Dev 

390.50635 

Std  Err  Mean 

16.994608 

Upper  95%  Mean 

7183.5336 

Lower  95%  Mean 

7116.7626 

N 

528 

41  -  Distributions  LOSS_RATE=FY13 


A  -  Sum(Mean(Strength)) 


6300  6600  6900  7200  7500  7800  8100  8400 


Summary  Statistics 

Mean 

7256.6818 

Std  Dev 

406.42045 

Std  Err  Mean 

17.68718 

Upper  95%  Mean 

7291.4279 

Lower  95%  Mean 

7221.9358 

N 

528 

41  -  Distributions  LOSS_RATE=FY14 


A 


w  Sum(Mean(Strength)) 


6500  6800  7100  7400  7700  8000  8300 


Summary  Statistics 

Mean 

7347.9519 

Std  Dev 

397.56849 

Std  Err  Mean 

17.301948 

Upper  95%  Mean 

7381.9411 

Lower  95%  Mean 

7313.9626 

N 

528 
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4  -  Distributions  L0SS_RATE=FY15 


»  Sum(Mean(Strength)) 

I - □□ 


6400  6700  7000  7B00  7600  7900  8200  8500 


4i  ▼  Summary  Statistics 

Std  Dev 

417.71115 

Std  Err  Mean 

18.178544 

Upper  95%  Mean 

7353.503 

Lower  95%  Mean 

7282.0804 

N 

528 

41  -  Distributions  L0SS_RATE=FY06 
41  ▼  Sum(Mean(Strength)) 


6700  6900  7100  7300  7500  7700  7900  8100  8300 


4l  *  Summary  Statistics 

7439.6864 
343.03445 
14.928658 
7469.0133 
7410.3594 
528 


Mean 
Std  Dev 
Std  Err  Mean 
Upper  95%  Mean 
Lower  95%  Mean 
N 


4l  -  Distributions  L0SS_RATE=FY07 
^  ~  Sum(Mean(Strength)) 


3 


6500  6700  6900  7100  7300  7500  7700  7900  8100 


^  ~  Summary  Statistics 


Mean 

7255.4235 

Std  Dev 

352.08836 

Std  Err  Mean 

1 5.322679 

Upper  95%  Mean 

7285.5245 

Lower  95%  Mean 

7225.3225 

N 

528 
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